Context-aware voice guidance

ABSTRACT

A context-aware voice guidance method is provided that interacts with other voice services of a user device. The voice guidance does not provide audible guidance while the user is making a verbal request to any of the voice-activated services. Instead, the voice guidance transcribes its output on the screen while the verbal requests from the user are received. In some embodiments, the voice guidance only provides a short warning sound to get the user&#39;s attention while the user is speaking on a phone call or another voice-activated service is providing audible response to the user&#39;s inquires. The voice guidance in some embodiments distinguishes between music that can be ducked and spoken words, for example from an audiobook, that the user wants to pause instead of being skipped. The voice guidance ducks music but pauses spoken words of an audio book in order to provide voice guidance to the user.

CLAIM OF BENEFIT TO PRIOR APPLICATIONS

This is a continuation of U.S. Non-Provisional patent application Ser.No. 13/632,121, filed Sep. 30, 2012, which claims the benefit of U.S.Provisional Patent Application 61/655,995, filed Jun. 5, 2012; U.S.Provisional Application 61/655,997, filed Jun. 5, 2012; U.S. ProvisionalPatent Application 61/656,015, filed Jun. 6, 2012; U.S. ProvisionalApplication 61/656,032, filed Jun. 6, 2012; U.S. Provisional Application61/656,043, filed Jun. 6, 2012; U.S. Provisional Patent Application61/656,080, filed Jun. 6, 2012; U.S. Provisional Application 61/657,864,filed Jun. 10, 2012; U.S. Provisional Application 61/657,880, filed Jun.10, 2012; U.S. Provisional Patent Application 61/699,842, filed Sep. 11,2012; U.S. Provisional Application 61/699,855, filed Sep. 11, 2012; andU.S. Provisional Patent Application 61/699,851, filed Sep. 11, 2012. Theabove-mentioned applications are incorporated herein by reference.

BACKGROUND

Many map-based applications available today are designed for a varietyof different devices (e.g., desktops, laptops, tablet devices,smartphones, handheld global positioning system (GPS) receivers, etc.)and for various different purposes (e.g., navigation, browsing, sports,etc.). Most of these applications generate displays of a map based onmap data that describes relative locations of streets, highways, pointsof interest, etc., in the map.

The maps used in such applications are usually two-dimensional (2D) mapsor three-dimensional (3D) maps. However, a large number of theapplications use 2D maps due in part to the processing-intensive demandsof viewing 3D maps. For the same reason, the applications that use 3Dmaps are often slow, inefficient, plain, and/or simple, to the pointthat renders the application useless.

BRIEF SUMMARY

Some embodiments of the invention provide a device that includes anavigation application with several novel features. In some embodiments,the device has a touch-sensitive screen that displays the output of theapplication, and a multi-touch interface that allows a user to providetouch and gestural inputs through the screen to interact with theapplication.

In some embodiments, the novel features of the navigation applicationinclude (1) multiple different views (e.g., a two-dimensionalturn-by-turn view, a three-dimensional turn-by-turn view, an overallroute view, etc.) and smooth transitions between these views during thenavigation, (2) novel user interface (UI) controls for navigation, (3)realistic looking road signs for identifying maneuvers along a navigatedroute, (4) dynamic generation of instructions and directional indicatorsfor road signs and other presentations of the identified maneuvers, (5)informative navigation displays when the navigation application isoperating in the background on the device, (6) novel voice recognitionnavigation guidance, and (7) integration with other routing applicationsavailable on or for the device.

While all these features are part of the navigation application in someembodiments, other embodiments do not employ all of these features inthe navigation application. Also, in some embodiments, the navigationapplication is part of an integrated mapping application that providesseveral other useful operations, including location browsing, mapsearching, and route identifying operations. However, one of ordinaryskill will realize that in other embodiments, the navigation applicationis a stand-alone application that does not include some or all of theseother operations.

Each of the above-described features are described here. As mentionedabove, the navigation application of some embodiments provides multipledifferent views during navigation and smooth transitions between theseviews. In some embodiments, examples of such views include atwo-dimensional (2D) turn-by-turn view, a three-dimensional (3D)turn-by-turn view, and an overall route view. The application in someembodiments generates the turn-by-turn views from a perspectiverendering position within a 3D navigation scene that the device renders.This perspective rendering position in some embodiments is adjustableand can be viewed as a virtual camera that can capture the 3D navigationscene from a variety of different perspectives (e.g., from a variety ofdifferent positions and orientations). Accordingly, in some embodiments,the turn-by-turn navigation is an animated rendering of navigated routethat is rendered from the vantage point of a virtual camera thattraverses along the direction of the route based on the traversaldirection and speed of the user carrying the device, which in someembodiments is captured by directional data (e.g., GPS data,triangulated cell-tower data, etc.) associated with the device.

During navigation, the navigation application of some embodiments allowsa user to change the position of the virtual camera (i.e., the positionfrom which the navigated route is rendered) through gestural input onthe device's screen. Movement of the virtual camera (i.e., movement ofthe position from which the route is rendered) allows the navigationapplication to present alternative 3D view. Some embodiments even usethe virtual camera to render a top-down 2D view for the turn-by-turnnavigation, while other embodiments render the top-down 2D view byzooming in and out of a 2D map.

In some embodiments, the navigation application presents a 3D control(e.g., button) that serves both as a 3D indicator and a 3Dinitiator/toggle. The 3D control is implemented in some embodiments as afloating control that can “float” above the 2D or 3D navigationpresentation when it is needed and “float” out of the presentation whenit is not needed. This control also serves as an indicator that thecurrent view is a 3D view. The 3D control may have different appearances(e.g., colored as grey, black, blue, etc.) to provide differentindications. In some embodiments, the 3D control is grey when 3D data isnot available for the user's current location, black when the 3D data isavailable but the user is currently viewing the map in 2D, and purplewhen the user is viewing the map in 3D mode. In some embodiments, the 3Dcontrol displays an image of a building when the user is at a certainzoom level and provides a “flyover” of the buildings in the area whenselected by the user. It also provides a quick mechanism of getting intoand out of 3D navigation. As further described below, the navigationapplication allows transitions between the 2D and 3D navigation viewsthrough other gestural inputs of the multi-touch interface of thedevice.

The navigation application in some embodiments uses floating controls inorder to keep the on-screen controls to a minimum and thereby display asmuch of the interactive navigation as possible. In some embodiments, thefloating controls are part of a cluster of controls that adapt to thetask at hand by adjusting its contents in an animated fashion when auser moves between different navigation views, or between differentapplication modalities for embodiments in which the navigation is justone of several modalities of another application. This adaptive natureallows the navigation application to optimize for different tasks whilemaintaining a consistent look and interaction model while moving betweenthose tasks.

When the navigation application starts a navigation presentation, theapplication in some embodiments (1) automatically hides the floatingcontrols and a bar (containing other UI controls) on the top of a mapalong which the navigation is displayed, and (2) starts a full-screenturn-by-turn navigation presentation. In this mode, the applicationrestricts touch interaction with the map. In some embodiments, a tap isrequired to access the controls that were automatically hidden. In someembodiments, these controls are adapted towards a full-screen navigationlook, including a prominent display of the estimated time of arrival(ETA) in the bar along the top.

In some embodiments, one of the controls in the top bar is an overviewbutton. By selecting this button at any time during the navigation, auser can seamlessly switch between the full-screen; turn-by-turnpresentation that displays a view optimized for turn-by-turn directions;and an overview presentation that displays a view of the remaining routethat better accommodate browsing.

In some embodiments, the constant set of controls and the in-placetransition in the map provide continuity between the overview mode andthe full-screen mode. These controls also include a control that allowsthe user to end the navigation in either the overview mode orfull-screen model. Some embodiments also allow for a search to beperformed while navigating. For instance, some embodiments provide apull down handle that allows the search field to be pulled into theoverview display while navigating in the overview mode. Alternatively,or conjunctively, some embodiments allow for searches to be performedduring navigation through a voice-recognition input of the device ofsome embodiments. Also, in some embodiments, the application allows auser to perform searches (e.g., voice-initiated and/or text-basedsearches) during turn-by-turn navigation. The navigation application ofsome embodiments also allows navigation to be initiated throughvoice-recognition input of the device.

During navigation, the navigation application of some embodiments alsoallows a user to provide some gestural input without reference to thefloating controls or the top-bar controls. For instance, differentembodiments provide different gestural inputs to adjust the 2D/3D viewduring turn-by-turn navigation. In some embodiments, the gestural inputis a two-finger pinching/spreading operation to adjust the zoom level.This adjustment of the zoom level inherently adjusts the position androtation of the camera with respect to the route direction, and therebychanges the 2D/3D perspective view of the route direction.Alternatively, other embodiments provide other gestural inputs (e.g., afinger drag operation) that change the position of the camera instead ofor in addition to the zoom operation. In yet other embodiments, agestural input (e.g., a finger drag operation) momentarily changes theviewing direction of the camera to allow a user to momentarily glance toa side of the navigated route. In these embodiments, the applicationreturns the camera to its previous view along the route after a shorttime period.

Another novel feature of the navigation application are therealistic-looking road signs that are used during navigation. In someembodiments, the signs are textured images that bear a strongresemblance to actual highway signs. These signs in some embodimentsinclude instructional arrows, text, shields, and distance. Thenavigation application of some embodiments presents a wide number ofsign variants in a large number of different contexts. Also, in someembodiments, the application presents signs in different colorsaccording to the regional norms.

For maneuvers that are close together, the application in someembodiments presents a secondary sign beneath the primary sign. Also, asone maneuver is passed, the navigation application animates the signpassing away with a motion that mimics a sign passing overhead on thehighway. When an upcoming maneuver is approaching, the navigationapplication draws attention to the sign with a subtle animation (e.g., ashimmer across the entire sign).

In some embodiments, the navigation application dynamically generatesinstructions for a road sign and other presentation (e.g., a list view)associated with a navigation maneuver based on the context under whichthe application is displaying the sign or presentation. For a givencontext, the instruction text is chosen by considering factors such asthe available space, the availability of information conveyed by meansother than text (e.g., the availability of voice guidance), thelocalized length of each of the instruction variants, the size of thedisplay screen of the device, etc. By locally synthesizing andevaluating several alternatives, the application can pick an optimalinstruction string in every scenario.

Similarly, the navigation application of some embodiments adaptivelygenerates directional graphical indicators for a road sign and otherpresentation (e.g., a list view) associated with a navigation maneuverbased on the context under which the application is displaying the signor presentation. For instance, when there is sufficient space on a signor presentation for the use of a bigger sign, the navigation applicationof some embodiments identifies a maneuver to perform at a juncture alonga route by using a larger graphical directional indicator that includes(1) a prominent stylized arrow roughly representing the path of thevehicle, and (2) a de-emphasized set of lines and curves correspondingto other elements of the junction. In some embodiments that use thisapproach, a right turn at a T-junction is represented by a large arrowwith a right-angle joined with a smaller, dimmer segment that runsparallel to one of the large arrow's segments. The smaller segment insome embodiments is also pushed off to the side so that the path takenby the vehicle dominates.

Such a representation of a maneuver (that includes a prominent stylizedarrow and a de-emphasized set of lines) provides fairly completeinformation about the maneuver while remaining abstract and easilyunderstandable. However, there may not be sufficient space on the signor other presentation for such a representation in other contexts.Accordingly, for such cases, the navigation application of someembodiments uses an alternate representation of the maneuver that omitsdisplaying the junction and instead only displays an arrow in thedirection of movement.

To generate either the prominent stylized arrow or the simplified arrowfor a juncture maneuver along a route, the navigation application insome embodiments receives from a server a description of the junctureand maneuver. In some embodiments, the server performs an automatedprocess to generate this description based on map data, and providesthis information in terms of compressed, geometric point data. Also, atthe beginning of a route navigation, the server in some embodimentssupplies to the navigation application the description of all juncturesand maneuvers along the route, and occasionally updates this descriptionwhen the user strays from the route and the server computes a new route.

When the navigation application receives the juncture and maneuverdescription, the application of some embodiments initially performs aprocess to simplify the characterization of the juncture and themaneuver, and then uses this simplified characterization to generate theprominent stylized graphical directional indicator for the juncture. Todisplay a maneuver at a juncture, some navigation applications oftenprovide a plain arrow that is not expressed in terms of the juncture anddoes not convey much information, while other navigation applicationsprovide a very detailed representation of the juncture and a complexdirectional representation through this detailed representation. Thus,one existing approach provides very little information, while anotherapproach provides so much information that the information is renderedpractically useless. By generating the prominent stylized directionalindicator based on the simplified description of the juncture, thenavigation application of some embodiments displays a detailedrepresentation of the maneuver at the juncture while eliminating some ofthe unnecessary complexities of the juncture.

In some embodiments, the navigation application provides navigationinstructions while the application is operating in the background andeven while the device is locked. In some embodiments, the device islocked when only a reduced set of controls can be used to provide inputinto the device. For instance, in some embodiments, the locking of thedevice greatly limits the number of inputs that a user can providethrough the touch-sensitive screen of the device.

In some embodiments, voice guidance instructions are one example ofinstructions that can be provided while the navigation application isoperating in the background or while the device is locked. Alternativelyto, or conjunctively with, the voice guidance, the navigationapplication can provide text and/or graphical instructions in at leasttwo modes while operating in the background.

First, the application of some embodiments incorporates in the lockscreen background, a live navigation view (e.g., a turn-by-turn view)that includes text and graphical navigation description in thelock-screen display. With this presentation, the user can see thenavigation instructions while the application is running in thebackground without unlocking the device. In some embodiments, theapplication further refines the lock screen experience by sendingnotifications that would normally occupy the space being taken by thenavigation display to a drawer in the lock-screen display, which in someembodiments is done immediately while in other embodiments is done aftera short time period in which the notification is shown on the lockscreen view. Also, whenever a user unlocks the device, some embodimentsreturn without animation to the navigation display in order to make theexperience seamless.

In some embodiments, the application turns off the lock screennavigation display after a time period if no maneuvers are impending.However, the application in some of these embodiments lights up thescreen when approaching an imminent maneuver and/or new navigationinstructions need to be provided. This is a small amount of timerelative to the duration of each step, so the display of the navigationinstructions does not come at the expense of noticeably degraded batterylife. To enhance the experience, the navigation application in someembodiments activates an ambient light sensor well before the navigationprompt so that the ambient light settings can be used to light thescreen to the correct brightness when it comes time to show thenavigation map.

Second, in some embodiments, the navigation application operates in thebackground even when the device is unlocked. This is the case when thenavigation application operates on a device (e.g., a smartphone) thatexecutes several other applications. In such a device, the navigationapplication would operate in the background when the device ispresenting a view (e.g., a page) that is provided by the operatingsystem of the device or a view that is provided by another applicationon the device.

When the navigation application operates in the background on anunlocked device, the device in some embodiments (1) uses a double-heightstatus bar to indicate the background operation of the navigationapplication when far from an upcoming maneuver, and (2) uses a sign-likenavigation banner that includes dynamically updated distance to amaneuver when approaching a maneuver or when guidance instructions areaudible. Further, the application maintains the sign-like banner untilthe maneuver is complete and suppresses other notifications in thatspace. Selection of either the double-height status bar or thenavigation banner in some embodiments directs the device to switch to anavigation view generated by the navigation application.

The above-described features as well as some other features of thenavigation application of some embodiments are further described below.In the description above and below, many of the features are describedas part of an integrated mapping application that provides novellocation browsing, location searching, route identifying and routenavigating operations. However, one of ordinary skill will realize thatthese novel operations are performed in other embodiments byapplications that do not perform all of these operations, or performother operations in addition to these operations.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description and the Drawings is needed.Moreover, the claimed subject matters are not to be limited by theillustrative details in the Summary, Detailed Description and theDrawings, but rather are to be defined by the appended claims, becausethe claimed subject matters can be embodied in other specific formswithout departing from the spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purposes of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates an example of a device that executes an integratedmapping application of some embodiments of the invention.

FIG. 2 illustrates an example in terms of three stages of a user'sinteraction with the mapping application to obtain routing directions.

FIG. 3 illustrates how the navigation application of some embodimentsprovides the 3D control as a quick mechanism for entering a 3Dnavigating mode.

FIG. 4 illustrates a device that displays a mapping application as theapplication transitions from a non-immersive map view for map browsinginto an immersive map view for navigation.

FIG. 5 presents a simplified example to illustrate the concept of avirtual camera.

FIG. 6 illustrates that the mapping application of some embodimentschanges the appearance of the 3D control to indicate different 2D and 3Dstates of the map view.

FIG. 7 illustrates switching from 3D mode to 2D mode in someembodiments.

FIG. 8 illustrates the adjustment of the distance of a virtual camera bycontracting and expanding gestures.

FIG. 9 illustrates an embodiment of a camera whose angle can be adjustedby gestures.

FIG. 10 conceptually illustrates a feature provided by the mappingapplication of some embodiments for maintaining the position of avirtual camera within a defined range along an arc.

FIG. 11 illustrates a full screen mode of some embodiments.

FIG. 12 illustrates the navigation application with the controls hiddenand revealed during a phone call on the device in some embodiments.

FIG. 13 illustrates the end of a programmed route in some embodiments.

FIG. 14 illustrates a navigation program ending control in someembodiments.

FIG. 15 illustrates the rotation of a map when a user pushes it sidewaysin some embodiments.

FIGS. 16 and 17 illustrate overview controls in some embodiments.

FIG. 18 conceptually illustrates a processing, or map rendering,pipeline performed by the mapping application of some embodiments inorder to render a map for display at the client device.

FIGS. 19A and 19B conceptually illustrate a state diagram that describesdifferent states and transitions between these states of the integratedmapping, search, and navigation application of some embodiments (e.g.,the application described in the above sections).

FIG. 20 illustrates several GUI scenarios in which such highway shieldsare used in some embodiments.

FIG. 21 illustrates, over four stages, the animation of some embodimentsfor removing a navigation sign and introducing the next sign.

FIG. 22 illustrates such a shimmer animation over four stages thatillustrate the background of the display as gray, in order to contrastwith the shimmer as it moves across the sign in some embodiments.

FIG. 23 illustrates the display of two signs for maneuvers in quicksuccession over four stages in some embodiments.

FIG. 24 conceptually illustrates an operation performed by a mappingservice of some embodiments to generate a route for a requesting deviceand provide the route, with navigation instructions, to the requestingdevice.

FIG. 25 conceptually illustrates a process performed by the mappingservice of some embodiments in order to generate and transmit route andintersection data to a user.

FIG. 26 conceptually illustrates a process of some embodiments fordetermining path segments between sets of junctions that should betreated together as single intersections.

FIG. 27 illustrates an example of a junction and shows that there is norequirement that the path segments meet at right angles or that thepaths continue in a straight line through the junction in someembodiments.

FIG. 28 illustrates an intersection that includes two dual carriagewaypaths and a one-way road in some embodiments.

FIG. 29 conceptually illustrates a process for linking together severaljunctions into a single intersection and identifying the branches of theintersection in some embodiments.

FIG. 30 illustrates a commonly existing intersection, between a dualcarriageway with two paths and a dual carriageway with two paths in someembodiments.

FIG. 31 illustrates an intersection in which left-turn channels aredefined as separate path segments in some embodiments.

FIG. 32 illustrates a slip road in an intersection in some embodiments.

FIG. 33 illustrates a slip road in an intersection in some embodiments.

FIG. 34 illustrates additional two-way path in an intersection in someembodiments.

FIG. 35 illustrates the reduction of an eight-path intersection intofour branches in some embodiments, in which the angle of the rightbranch is at half the offset from horizontal as the right exit path,because the right entrance path is on the horizontal.

FIG. 36 illustrates the reduction of a different eight-path intersectioninto five branches in some embodiments.

FIG. 37 conceptually illustrates an example of a data structure of someembodiments for a point type intersection.

FIG. 38 illustrates a data structure of some embodiments for aroundabout intersection.

FIG. 39 conceptually illustrates the reduction of a roundaboutintersection to intersection data in some embodiments.

FIG. 40 conceptually illustrates a process of some embodiments formodifying intersection data in order to provide navigation informationfor a route.

FIG. 41 illustrates a conceptual drawing of a route taken through anintersection a data structure for the intersection, and the modificationof the data structure to create a new data structure for turn-by-turnnavigation instructions.

FIG. 42 illustrates several different scenarios in which the mappingapplication displays different types of graphical indicator arrows tovisually represent maneuvers to a user in some embodiments.

FIG. 43 illustrates several scenarios for the same turn, and how thedifferent arrows might be used for the same turn in some embodiments.

FIG. 44 conceptually illustrates a process of some embodiments fordisplaying graphical indicators during route inspection.

FIG. 45 conceptually illustrates a process of some embodiments thatperforms navigation over such a route.

FIG. 46 conceptually illustrates a process that generates such graphicaldirectional indicators for the maneuvers of a route.

FIG. 47 conceptually illustrates a process that attempts to set theangles of branches of a juncture along a route to multiples of apre-specified angle in some embodiments.

FIG. 48 illustrates a particular juncture situation in some embodiments.

FIG. 49 illustrates two examples where default juncture/maneuverindicators are used instead of the geometry-based indicators in someembodiments.

FIG. 50 illustrates an example of a roundabout in which the simplifiedgeometry is not used by some embodiments.

FIG. 51 conceptually illustrates a mapping application of someembodiments that generates directional indicators for differentcontexts.

FIG. 52 illustrates an example of the synthesis of differentinstructions for a particular maneuver at a juncture according to someembodiments.

FIG. 53 illustrates several different scenarios in which the mappingapplication displays different examples of the adaptive instructions forthe particular maneuver of the first juncture in a variety of differentsituations.

FIG. 54 illustrates additional scenarios in which the mappingapplication uses the synthesized instruction sets in some embodiments.

FIG. 55 conceptually illustrates a process of some embodiments fordisplaying text instructions during route inspection.

FIG. 56 conceptually illustrates a process of some embodiments thatperforms navigation over such a route.

FIG. 57 conceptually illustrates a process of some embodiments fordecoding encoded juncture data and synthesizing instruction elementsfrom the route and juncture data received from a mapping service.

FIG. 58 conceptually illustrates a process of some embodiments forgenerating navigation instruction variants for display in differentcontexts.

FIG. 59 conceptually illustrates a system architecture that includesmapping and navigation application of some embodiments that generatestext instructions for different contexts.

FIG. 60 illustrates a user device display when navigation is in processin background in some embodiments of the invention.

FIG. 61 conceptually illustrates a process of some embodiments forproviding directions while a navigation application is running in thebackground.

FIG. 62 illustrates a user interface of some embodiments in whichnavigation instructions are given while the navigation application isrunning in the background of another application.

FIG. 63 illustrates a navigation bar displayed at the top of anapplication in some embodiments.

FIG. 64 illustrates the user interface of a device in some embodimentswhere the device reaches its destination while the navigationapplication is running in the background of another application.

FIG. 65 illustrates interaction between a call status bar and anavigation instruction bar.

FIG. 66 illustrates a device of some embodiments that enters locked modewith the navigation application running in the background and exitslocked mode with the navigation application running in the foreground.

FIG. 67 illustrates a device in some embodiments that enters locked modewith the navigation application running in the foreground and exits thelocked mode with the navigation application still running in theforeground.

FIG. 68 illustrates a navigation application giving directions on alocked device in some embodiments of the invention.

FIG. 69 illustrates the locked mode view of some embodiments when thedevice reaches its destination.

FIG. 70 illustrates a locked view notification system of someembodiments.

FIG. 71 illustrates the viewing of notification messages after unlockinga device in some embodiments of the invention.

FIG. 72 illustrates a process for switching the device screen on whenapproaching a navigation point in some embodiments of the invention.

FIG. 73 illustrates multiple stages that a device goes through when nocommands are given to it while a navigation application runs in thebackground in some embodiments of the invention.

FIG. 74 conceptually illustrates a process of some embodiments forturning on the screen when a notification message is received.

FIG. 75 conceptually illustrates a process for performingvoice-activated interactions with an interactive map in some embodimentsof the invention.

FIG. 76 illustrates a user device when lock-screen is not active in someembodiments of the invention.

FIG. 77 illustrates a user device with lock-screen active in someembodiments of the invention.

FIG. 78 conceptually illustrates a process for providing voice-activatednavigation while lock-screen is activated in some embodiments of theinvention.

FIG. 79 conceptually illustrates a process for receiving a naturallanguage utterance and retrieving and presenting the user's currentnavigation status while the user is traveling along a route in someembodiments of the invention.

FIG. 80 illustrates a user device when natural language utterances areused during voice-activated navigation in some embodiments of theinvention.

FIG. 81 illustrates a user device when natural language utterances areused during voice-activated navigation in some embodiments of theinvention.

FIG. 82 illustrates user device of FIG. 80 after the user makes aninquiry based on the current dialog.

FIG. 83 illustrates user device of FIG. 80 after the user makes aninquiry based on the current dialog.

FIG. 84 conceptually illustrates a process for providing voice-activatedsearch and navigation in some embodiments of the invention.

FIGS. 85A-85D illustrate 12 stages of a user interface of someembodiments in which a user is using the voice-activated service tosearch for points of interest and destinations.

FIG. 86 conceptually illustrates an alternative process for providingvoice-activated search and navigation in some embodiments of theinvention.

FIG. 87 illustrates a user device during navigation in some embodimentsof the invention.

FIG. 88 illustrates a user device during navigation in some embodimentsof the invention.

FIG. 89 illustrates the user device of FIG. 87 when the user does notlike to select the first coffee shop.

FIGS. 90A-90E conceptually illustrate portions of the voice-activatedservice of some embodiments of the invention that are used during asearch operation.

FIG. 91 illustrates 4 stages of a user interface of some embodimentswhere navigation is incorporated into voice-activated service output insome embodiments of the invention.

FIG. 92 conceptually illustrates a process used by the voice-activatedservice to incorporate navigation output in some embodiments of theapplication.

FIG. 93 conceptually illustrates a process that provides different typesof audible prompts on an electronic device based on the audio sessionscurrently active on the device in some embodiments of the invention.

FIG. 94 conceptually illustrates a process for providing audible promptswhile another audio session is active in some embodiments of theinvention.

FIG. 95 conceptually illustrates a more detailed process for providingdifferent type of audible prompts on an electronic device in someembodiments of the invention.

FIG. 96 conceptually illustrates a process for providing non-verbalprompts with directionality information in some embodiments of theinvention.

FIG. 97 illustrates a user device when a navigation session and a phonecall are in progress in some embodiments of the invention.

FIG. 98 illustrates another example of providing directional informationby non-verbal prompts in some embodiments of the invention.

FIG. 99 illustrates an example of providing directional information bypanning non-verbal prompts in some embodiments of the invention.

FIG. 100 illustrates a user interface for setting relative prompt levelsof the navigation application in some embodiments of the invention.

FIG. 101 conceptually illustrates a process for providing verbalnavigation prompts while another audio session is active in someembodiments of the invention.

FIG. 102 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 103 illustrates providing a voice prompt on a device while therelative volume level of the voice prompt is set high and the contentthat is inherently loud is being played in some embodiments of theinvention.

FIG. 104 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 105 illustrates providing a voice prompt on a device when therelative prompt volume level is set to low and the content that isinherently soft is being played in some embodiments of the invention.

FIG. 106 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 107 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 108 illustrates providing a voice prompt on a device where therelative volume level of the prompt is set to normal and the content isrecorded at a similar loudness level as the voice prompt in someembodiments of the invention.

FIG. 109 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 110 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention.

FIG. 111 conceptually illustrates a process for playing a voice promptin some embodiments of the invention while another audio session isactive.

FIG. 112 illustrates providing a voice prompt on a device while acontent that can be paused is being played in some embodiments of theinvention.

FIG. 113 conceptually illustrates a high-level software architecture ofthe context aware guidance system of some embodiments.

FIG. 114 is an example of an architecture of a mobile computing deviceof some embodiments.

FIG. 115 conceptually illustrates an example of an electronic systemwith which some embodiments of the invention are implemented.

FIG. 116 illustrates a map service operating environment, according tosome embodiments.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

I. Navigation User Interface

A. Start

The navigation application of some embodiments is part of an integratedmapping application that includes several useful modalities, includinglocation browsing, map searching, route identifying and route navigatingoperations. This integrated application (referred to below as themapping application, the navigation application, or the integratedapplication) in some embodiments is defined to be executed by a devicethat has a touch-sensitive screen that displays the output of theapplication. In some embodiments, this device has a multi-touchinterface for allowing a user to provide touch and gestural inputsthrough the screen to interact with the application. Examples of suchdevices are smartphones (e.g., iPhone® sold by Apple Inc., phonesoperating the Android® operating system, phones operating the Windows 8®operating system, etc.).

FIG. 1 illustrates an example of a device 100 that executes anintegrated mapping application of some embodiments of the invention.This figure also illustrates an example of launching a route navigationin this application. This application has a novel user interface (UI)design that seamlessly and cohesively integrates the controls for eachof its different modalities by using a minimum set of on-screen controlsthat float on top of the content in order to display as much of thecontent as possible. Additionally, this cluster adapts to the task athand, adjusting its contents in an animated fashion when a user movesbetween the different modalities (e.g., between browsing, searching,routing and navigating). This common element with an adaptive natureenables the mapping application to optimize for different tasks whilemaintaining a consistent look and interaction model while moving betweenthose tasks.

FIG. 1 shows six stages 105, 110, 115, 117, 119, 121 of interaction withthe mapping application. The first stage 105 shows the device's UI 120,which includes several icons of several applications in a dock area 125and on a page of the UI. One of the icons on this page is the icon forthe mapping application 130. The first stage shows a user's selection ofthe mapping application through touch contact with the device's screenat the location of this application on the screen.

The second stage 110 shows the device after the mapping application hasopened. As shown in this stage, the mapping application's UI has astarting page that in some embodiments displays (1) a map of the currentlocation of the device and (2) several UI controls arranged in a top bar140, and as floating controls. As shown in FIG. 1, the floating controlsinclude an indicator 145, a 3D control 150, and a page curl control 155,while the top bar 140 includes a direction control 160, a search field165, and a bookmark control 170.

In some embodiments, a user can initiate a search by tapping in thesearch field 165. This directs the application to present an animationthat (1) presents an on-screen keyboard and (2) opens a search tablefull of invaluable completions. This table has some importantsubtleties. When the search field is tapped and before the terms areedited, or when the search field is empty, the table contains a list of“recents,” which in some embodiments are recent searches and routedirections that the user has requested. This makes it very easy toquickly bring up recently accessed results.

After any input on the search field, the table is filled with searchcompletions both from local sources (e.g., bookmarks, contacts, recentsearches, recent route directions, etc.) and remote servers. Theincorporation of the user's contact card into the search interface addsadditional flexibility to the design. When showing recents, a route fromthe current location to the user's home is always offered in someembodiments, while it is offered in the contexts that are deemed to be“appropriate” in other embodiments. Also, when the search term matchesat least part of an address label (e.g., ‘ork’ for ‘Work’), theapplication presents the user's labeled address as a completion in thesearch table in some embodiments. Together these behaviors make thesearch UI a very powerful way to get results onto a map from a varietyof sources. In addition to allowing a user to initiate a search, thepresence of the text field in the primary map view in some embodimentsalso allows users to see the query corresponding to search results onthe map and to remove those search results by clearing the query.

The bookmark control 170 (e.g., button) allows location and routes to bebookmarked by the application. The position indicator 145 allows thecurrent position of the device to be specifically noted on the map. Oncethis indicator is selected, the application maintains the currentposition of the device in the center of the map. In some embodiments, itcan also identify the direction to which the device currently points.

The 3D control 150 is a control for viewing a map or inspecting a routein three dimensions (3D). The mapping application provides the 3Dcontrol as a quick mechanism of getting into and out of 3D. This controlalso serves as (1) an indicator that the current view is a 3D view, (2)an indicator that a 3D perspective is available for a given map view(e.g., a map view that is zoomed out might not have a 3D viewavailable), (3) an indicator that a 3D perspective is not available(e.g., the 3D data is not available for the map region), and (4) anindicator that a flyover animation is available at the given zoom level.The 3D control may provide a different appearance corresponding to eachindication. For instance, the 3D control may be colored grey when the 3Dview is unavailable, black when the 3D view is available but the map isin the 2D view, and blue when the map is in the 3D view. In someembodiments, the 3D control changes to an image of a building when theflyover animation is available for the user's given zoom level andlocation on the map.

The page curl control 155 is a control that allows the application tominimize the number of on-screen controls, by placing certain lessfrequently used actions in a secondary UI screen that is accessiblethrough the page curl control that is displayed on the map. In someembodiments, the page curl is permanently displayed on at least some ofthe map views that the application provides. For instance, in someembodiments, the application displays the page curl permanently on thestarting page (illustrated in the second stage 110) that it provides forallowing a user to browse or search for a location or to identify aroute.

The direction control 160 opens a direction entry page 180 through whicha user can request a route to be identified between a starting locationand an ending location. The third stage 115 of FIG. 1 illustrates thatthe selection of the direction control 160 opens the direction entrypage 180, which is shown in the fourth stage 117. The direction controlis one of three mechanisms through which the mapping application can bedirected to identify and display a route between two locations; the twoother mechanisms are (1) a control in an information banner that isdisplayed for a selected item in the map, and (2) recent routesidentified by the device that are displayed in the search field 165.Accordingly, the information banner control and the search field 165 aretwo UI tools that the application employs to make the transition betweenthe different modalities seamless.

The fourth stage 117 shows that the direction entry page 180 includesstarting and ending fields for providing starting and ending locationsfor a route, and a table that lists recent routes that the applicationhas provided to the user. Other controls on this page are controls forstarting a route, for reversing the order of the start and endlocations, for canceling the direction request, for picking walking,auto, or public transit routes. These controls and other aspects of themapping application are described in U.S. patent application Ser. No.13/632,102, filed 30, 2012, entitled “Problem Reporting in Maps.” ThisU.S. patent application Ser. No. 13/632,102 is incorporated herein byreference.

The fourth stage illustrates the user selecting one of the recentdirections that was auto-populated in the table 182. The fifth stage 119then shows three routes on a 2D map view between the specified start andend locations specified through the page 180. It also shows theselection of the second route and some information about this route in abar at the top of the layout. This bar is shown to include start and endbuttons. The start button is shown to be selected in the fifth stage.

As shown in the sixth stage, the selection of the start button directsthe application to enter a turn-by-turn navigation mode. In thisexample, the application has entered a 2D turn-by-turn navigation mode.In other embodiments, the application will enter by default into a 3Dturn-by-turn navigation mode. In this mode, the application displays arealistic sign 184 that identifies the distance from the currentlocation of the device to the next juncture maneuver in the navigatedroute and some other pertinent information. The application alsodisplays a top bar that includes some information about the navigationas well as End and Overview buttons, for respectively ending thenavigation and obtaining an overview of the remaining portion of thenavigated route or the entire portion of the navigated route in otherembodiments.

The mapping application of some embodiments identifies the location ofthe device using the coordinates (e.g., longitudinal, altitudinal, andlatitudinal coordinates) in the GPS signal that the device receives atthe location of the device. Alternatively or conjunctively, the mappingapplication uses other methods (e.g., cell tower triangulation) tocompute the current location. When the user carrying the device deviatesfrom the route, the mapping application of some embodiments tracks thelocation of the device and re-calculates a new route from the deviatedlocation in order to re-direct the user to the destination location fromthe deviated location. In other words, the mapping application of someembodiments operating in the navigation mode requires the device to beon a route at all times.

The application further displays the floating 3D control and thefloating list control, which were described above. It should be notedthat the list control was adaptively added to the floating controlcluster upon entering the route inspection and route navigationmodalities, while the position indicator was removed from the floatingcontrol upon entering the route navigation modality. Also, upontransition from the route inspection mode to the route navigation mode,the application performs an animation in some embodiments that involvesthe page curl uncurling completely before the application transitionsinto the navigation presentation.

In some embodiments, the animation transition includes removing the topbar, its associated controls and the floating controls from thenavigation presentation, and moving the sign 184 to the top edge of thepresentation a short time period after starting the navigationpresentation. As further described below, the application requires theuser tap on the navigated map to bring back the top bar, its controlsand the floating controls, and requires another tap to remove thesecontrols again from the map, in some embodiments. Other embodimentsprovide other mechanisms for viewing and removing these controls.

As another way of allowing the user to get navigation experience, themapping application of some embodiments provides a UI item in aninformational banner that appears by a pin that represents a point ofinterest (POI). FIG. 2 illustrates an example in terms of three stages205-215 of a user's interaction with the mapping application to obtainrouting directions. This example is provided in the context of using acar icon 230.

The first stage 205 illustrates a map in a 3D map view. As shown, a 3Dcontrol 250 appears highlighted to indicate that the map is in a 3D mapview. The first stage 205 also illustrates two informational banners forthe two pins for the search resulting from running a search with asearch query “Pizza” as shown. The user selects the car icon 230. Asmentioned above, the car icon 230 is for showing one or more routes tothe location that is represented by a pin with which the banner thatincludes the car icon 230 is associated. The banner 240 which includesthe car icon 230 also shows a brief description of the place, a starrating, and an arrow for launching a “stage” for the POI.

The second stage 210 illustrates the two routes, route 1 and route 2,that the mapping application of some embodiments shows in response tothe selection of the car icon 230 in the previous stage 205. The userhas selected route 1 as indicated by the highlight on the route 1. Theuser also selects the start button. As mentioned above, the start buttonin some embodiments is for starting the navigation according to theselected route.

The third stage 215 illustrates that the mapping application displays aninstruction sign 260, which is the sign for the first instruction. Themapping application has replaced the clear control 255 and the startbutton with an end button 270 and an overview control 275 in the top bar140. The end button is for ending the navigation of the route and theoverview control 275 is for showing the entire route in the map view byadjusting the zoom level of the displayed map if adjusting the zoomlevel is necessary to show the entire route. In some embodiments, themapping application displays in the top bar 140 the ETA, the amount oftime to get to the destination, and the remaining distance to thedestination as shown.

When the mapping application receives a selection of the end buttonwhile the mapping application is operating in the route inspection mode,the mapping application of some embodiments stops inspection of theselected route by going back to map browsing mode. The mappingapplication of some embodiments goes back to the map browsing mode byremoving the selected route from the map, putting back the page curl,and replacing the information and controls in the top bar with a set ofother controls including a direction control, a search field, and abookmark control. That is, the mapping application takes the appearanceof the UI page back to a UI page similar to the UI page shown in thefirst stage 205. The mapping application of some embodiments does notshift the map to another region when switching to the map browsing modefrom the inspection mode.

B. 2D and 3D Navigation

The navigation application of some embodiments can display navigation ineither a 2D mode or a 3D mode. As mentioned above, one of the floatingcontrols is the 3D control 250 that allows a user to view a navigationpresentation in three dimensions (3D). FIG. 3 illustrates how thenavigation application of some embodiments provides the 3D control 250as a quick mechanism for entering a 3D navigating mode. This figureillustrates this operation in three stages 305-315. The first stage 305illustrates the user selecting the 3D control 150 while viewing atwo-dimensional navigation presentation.

The second stage 310 illustrates the navigation presentation in themidst of its transition into a 3D presentation. As shown in this figure,the 3D control appears highlighted at this stage to indicate that thenavigation presentation has entered a 3D mode. As mentioned above, thenavigation application generates the 3D view of the navigated map insome embodiments by rendering the map view from a particular position inthe three dimensional scene that can be conceptually thought of as theposition of a virtual camera that is capturing the map view. Thisrendering is further described below by reference to FIG. 5.

The third stage 315 then illustrates the navigation presentation at theend of its transition into its 3D appearance. As shown by the differencebetween the heights of the buildings in the second and third stages, thetransition from 2D to 3D navigation in some embodiments includes ananimation that shows three-dimensional objects in the navigated mapbecoming larger. Generating such animation that shows objectsrising/falling and becoming larger/smaller is further described in theU.S. patent application Ser. No. 13/632,027, filed Sep. 30, 2012,entitled “Displaying 3D Objects in a 3D Map Presentation.” This U.S.patent application Ser. No. 13/632,027 is incorporated herein byreference.

Some embodiments use a cinematic transition from the 2D map view to the3D map view or vice versa. For instance, when the mapping applicationreceives a selection of the 3D control 250 while showing a startinglocation of a route, the mapping application begins from the 2D map viewand transitions smoothly from a first virtual camera view for the 2D toa new virtual camera 3D view that is more zoomed in and pointing in thedirection of the start of the route. In doing so, the virtual cameraperforms a combination of translation, zoom, and rotation operations inorder to reach the start of the route for navigation. That is, thevirtual camera moves in an arc and rotates upward as the camera movesdownward along the arc. Also, the mapping application may rotate the arcitself to align the virtual camera viewpoint to the initial road segmentof the route. In other words, the mapping application rotates the mapduring the cinematic transition.

FIG. 4 illustrates a device 400 that displays a mapping application asthe application transitions from a non-immersive map view for mapbrowsing into an immersive map view for navigation, over six stages405-430.

The first stage 405 illustrates a user selecting a quick-route buttonfor a location “Pizza Place” in order to generate a route from theuser's current location (near the center of the screen of device 400) tothe selected location. The second stage 410 illustrates the mappingapplication displaying a route 435 to reach the location “Pizza Place.”At the second stage 410, the user selects the “Start” UI control 440.Accordingly, the application begins entering navigation.

As shown at the third through sixth stages 415-430, some embodiments usea cinematic transition from the 2D (or 3D) non-immersive map view intothe 3D immersive map view. The application display begins from itscurrent state (that shown at 410) and transitions smoothly from thefirst virtual camera view to the new virtual camera view that is morezoomed in and pointing in the direction of the start of the route. Indoing so, the virtual camera may perform a combination of translation,zoom, and rotation operations in order to reach the start of the routefor navigation. As shown in these stages, the virtual camera moves androtates into its eventual location behind the navigation locationindicator (i.e., the puck) shown in the sixth stage 430.

Also, in some embodiments, the mapping application provides twodifferent types of 3D presentations—an immersive 3D presentation and anon-immersive 3D presentation. The immersive presentation in someembodiments not only displays more geometries but also displays moredetails for the geometries that are displayed in the non-immersivepresentation. The mapping application also provides smooth transitionsbetween the non-immersive and immersive presentations.

To achieve such smooth transitions and generate other novel effects, themapping application of some embodiments uses a novel image processingpipeline. This pipeline performs a variety of pre-load operations todownload, retrieve and/or decompress map tiles that may be needed for anavigation presentation, to prepare its rendering pipeline for itsrendering operations, and to prepare a duplicate pipeline to smoothlytransition between the immersive and non-immersive 3D presentations. Inorder to display immersive and non-immersive 3D map presentations, someembodiments have to generate a variety of tiles for client devices torender in order to generate roads, building, and surrounding scenery. Insome embodiments, examples of such tiles include road and building tilesused for non-immersive 3D presentations, and navigation and buildingtiles used for immersive 3D presentations. This pipeline is described inabove-incorporated U.S. patent application Ser. No. 13/632,102 filedSep. 30, 2012, entitled “Problem Reporting in Maps.” This pipeline isalso described in detail in the U.S. patent application Ser. No.13/632,040, filed Sep. 30, 2012, entitled “Virtual Camera for 3D Maps.”This U.S. patent application Ser. No. 13/632,040 is incorporated hereinby reference.

In some embodiments, the non-immersive and immersive viewing modes areviewing modes for viewing different 3D maps that have differentconstructs and/or geometries. For instance, the non-immersive viewingmode of some embodiments is for viewing a 3D map that includes roads,buildings, land cover, etc. The immersive viewing mode is for viewing amore detailed 3D map that includes the same or similar elements (e.g.,roads, buildings, land cover, etc.) as the 3D map for the non-immersiveviewing mode. However, this more detailed 3D map also includes higherdetail constructs (e.g., trees, foliage, sidewalks, medians, lanes ofroads, road asphalt, medians, cross walks, etc.) that provide a morerealistic and rich 3D map.

In addition, the non-immersive and immersive viewing modes may bedefined for viewing 3D maps at different ranges of zoom levels. Forexample, the non-immersive viewing mode of some embodiments is definedfor viewing a 3D map at low zoom levels (e.g., zoom levels 0-14) whilethe immersive viewing mode of some embodiments is defined for viewingthe 3D map at high zoom levels (e.g., zoom levels 16-21). The viewingmodes may be defined to view any number of different zoom levels indifferent embodiments. In some instances, the range of zoom levels ofthe immersive viewing mode are defined as higher zoom levels than, lowerzoom levels than, the same zoom levels as, or zoom levels that overlapwith the zoom levels defined for the non-immersive viewing mode. Theseviewing modes and other aspects of the mapping application are describedin the U.S. patent application Ser. No. 13/632,040 filed Sep. 30, 2012,entitled “Virtual Camera for 3D Maps.” This U.S. patent application Ser.No. 13/632,040 is incorporated herein by reference.

1. Virtual Camera

The navigation application of some embodiments is capable of displayingnavigation maps from multiple perspectives. The application can showmaps in three dimensions (3D) or in two dimensions (2D). The 3D maps aregenerated simulations of a virtual scene as seen by a virtual camera.FIG. 5 presents a simplified example to illustrate the concept of avirtual camera 512. When rendering a 3D navigation map, a virtual camerais a conceptualization of the position in the 3D map scene from whichthe device renders a 3D view of the scene. FIG. 5 illustrates a locationin a 3D navigation map scene 510 that includes four objects, which aretwo buildings and two intersecting roads. To illustrate the virtualcamera concept, this figure illustrates three scenarios, each of whichcorresponds to a different virtual camera location (i.e., a differentrendering position) and a different resulting view that is displayed onthe device.

The first stage 501 shows the virtual camera 512 at a first positionpointing downwards at an angle (e.g., a 30 degree angle) towards the 3Dscene 510. By rendering the 3D scene from the position and angle shownin stage 501 the application generates the 3D map view 518. From thisposition, the camera is pointing at a location that is a moving positionin front of the device. The virtual camera 512 is kept behind thecurrent location of the device. “Behind the current location” in thiscase means backward along the navigation application's defined path inthe opposite direction from the current direction that the device ismoving in.

The navigation map view 518 looks as though it was shot by a camera fromabove and behind the device's location indicator 516. The location andangle of the virtual camera places the location indicator 516 near thebottom of the navigation map view 518. This also results in the majorityof the screen being filled with the streets and buildings ahead of thepresent location of the device. In contrast, in some embodiments, thelocation indicator 516 is in the center of the screen, with half of thescreen representing things ahead of the device and the other halfrepresenting things behind the device. To simplify the figure, no roadsigns are depicted for the views 518, 528, and 538.

The second stage 502 shows the virtual camera 512 at a differentposition, pointing downwards towards the scene 510 at a larger secondangle (e.g., −45°). The application renders the scene 510 from thisangle, resulting in the 3D navigation map view 528. The buildings andthe roads are smaller than their illustration in the first navigationmap view 518. Once again the virtual camera 512 is above and behind thelocation indicator 516 in the scene 510. This again results in thelocation indicator appearing in the lower part of the 3D map view 528.The location and orientation of the camera also results again in themajority of the screen displaying things ahead of the location indicator516 (i.e., the location of the car carrying the device), which is whatsomeone navigating needs to know.

The third stage 503 shows the virtual camera 512 at a top-down view thatlooks downwards on a location in the 3D map scene 510 that was used torender the 3D views 518 and 528. The scene that is rendered from thisperspective is the 2D map view 538. Unlike the 3D rendering operationsof the first and second stages that in some embodiments are perspective3D rendering operations, the rendering operation in the third stage isrelatively simple as it only needs to crop a portion of the 2D map thatis identified by a zoom level specified by the application or the user.Accordingly, the virtual camera characterization in this situationsomewhat unnecessarily complicates the description of the operation ofthe application as cropping a portion of a 2D map is not a perspectiverendering operation.

At the third stage 503, the mapping application in some embodimentsswitches from rendering a 3D scene from a particular perspectivedirection to cropping a 2D scene when the camera switches from the 3Dperspective view to a 2D top-down view. This is because in theseembodiments, the application is designed to use a simplified renderingoperation that is easier and that does not generate unnecessaryperspective artifacts. In other embodiments, however, the mappingapplication uses a perspective rendering operation to render a 3D scenefrom a top-down virtual camera position. In these embodiments, the 2Dmap view that is generated is somewhat different than the map view 538illustrated in the third stage 503, because any object that is away fromthe center of the view is distorted, with the distortions being greaterthe further the object's distance from the center of the view.

The virtual camera 512 moves along different trajectories in differentembodiments. Two such trajectories 550 and 555 are illustrated in FIG.5. In both these trajectories, the camera moves in an arc and rotatesdownward as the camera moves upward along the arc. The trajectory 555differs from the trajectory 550 in that in the trajectory 555 the cameramoves backward from the current location as it moves up the arc.

While moving along one of the arcs, the camera rotates to maintain apoint ahead of the location indicator at the focal point of the camera.In some embodiments, the user can turn off the three dimensional viewand go with a purely two dimensional view. For example, the applicationof some embodiments allows a three dimensional mode to be turned on andoff by use of a 3D button 560. The 3D button 560 is essential to theturn-by-turn navigation feature, where it has a role as an indicator anda toggle. When 3D is turned off, the camera will maintain a 2Dnavigation experience, but when 3D is turned on, there may still be sometop-down perspectives when 3D viewing angles are not appropriate (e.g.,when going around a corner that would be obstructed in 3D mode).

2. 3D Control

FIG. 6 illustrates in six different stages 605-630 that the mappingapplication of some embodiments changes the appearance of the 3D controlto indicate different 2D and 3D states of the map view. The first stage605 illustrates that the mapping application is displaying a map and thefloating controls including the 3D control 150. The mapping applicationis displaying the map in 2D at a certain low zoom level (map has notbeen zoomed in much) as shown. The 3D control 150 is displayed using afirst appearance (e.g., grey letters “3D”) to indicate the 3D map datais not available at this particular zoom level. The first stage 605 alsoshows that the mapping application is receiving the user's gesturalinput to zoom in on the map (i.e., to increase the zoom level).

The second stage 610 shows that the mapping application is displayingthe map at a higher zoom level than it did at the previous stage 605.However, the 3D control 150 is maintaining the first appearance becausethe 3D map data is still not available even at this particular higherzoom level. The second stage 610 also shows that the mapping applicationis receiving another gestural input to zoom in on the map further.

The third stage 615 shows that the mapping application is displaying themap at a higher zoom level than it did at the previous stage 610. Themapping application has changed the appearance of the 3D control 150into a second appearance (e.g., “3D” in black letters) to indicate thatthe 3D map data is available at this zoom level. When the mappingapplication receives a selection of the 3D control 150, the mappingapplication of some embodiments would change the appearance of the 3Dcontrol 150 to a third appearance (e.g., “3D” in blue letters) anddisplay the map in 3D (e.g., by changing into a perspective view from astraight-down view for 2D). The third appearance therefore wouldindicate that the map is displayed in 3D. The third stage 615 shows thatthe mapping application is receiving yet another gestural input to zoomin the map even further to a higher zoom level. The third stage 615shows that the mapping application is displaying buildings in the map asgrey boxes at this zoom level.

The fourth stage 620 shows that the mapping application is displayingthe map at a higher zoom level than it did at the previous stage 615.The mapping application has changed the appearance of the 3D control 150into a fourth appearance (e.g., a building icon in a first color asshown) in order to indicate that 3D immersive map data for rendering animmersive 3D map view is available at this zoom level. The fourth stage620 also shows that the mapping application of some embodiments isreceiving a selection of the 3D control 150.

The fifth and sixth stages 625 and 630 show subsequent views (though notnecessarily successive views) that the mapping application providesafter it starts to provide a 3D immersive map view. The zoom level doesnot change between the fifth and sixth stages in some embodiments butthe height of the buildings in the map views increases to provide ananimation that conveys that the view is moving into the 3D immersive mapview from the 2D view. Also, from the fourth stage 620 to the fifthstage 625, the mapping application has changed the appearance of the 3Dcontrol into the fifth appearance (e.g., a building icon in a secondcolor as shown) in order to indicate that the map is displayed in the 3Dimmersive view.

3. Automatic Changing of Views

The application of some embodiments allows any particular virtual cameraangle to be used, not just the 30 degree and 60 degree angles specifiedhere. The application of some embodiments allows the user to set thedownward angle for the camera. The application of some embodimentsautomatically adjusts the angle of the camera for various reasons,(e.g., to keep a particular point of focus near the top of the screen).In still other embodiments, the navigation application automaticallysets the angle of the camera, but allows the user to override theautomatically set angle.

In some embodiments, when a device running the navigation application ina 3D mode is about to reach a junction with a turn, the navigationapplication switches to a 2D mode in order to enable the user to moreclearly identify the turn. FIG. 7 illustrates the switching from 3D modeto 2D mode of some embodiments. The figure is shown in five stages701-705. In stage 701, the application shows a navigation map in a 3Dview. The navigation box 710 shows a right turn in 50 feet. The map 712is in 3D as is the location identifier 714.

As the device approaches the junction in stage 702 (as indicated bynavigation box 720) the 3D map 712 switches to a 2D map 722 with thelocation indicator 724 in 2D as well. The mapping application alsochanges the appearance of the 3D control 150 to indicate that the map isnow in 2D. The map 722 remains in 2D as the device rounds the corner instage 703. As the device rounds the corner, the navigation box 730 withthe instructions “turn right into A St.” in stage 703 is replaced by thenavigation box 740 with the instructions “0.5 miles continue straight onA St.” in stage 704. The map remains in 2D in stage 704 until the cornerhas been fully navigated at which point, in stage 705, the map returnsto a 3D view with new instructions “0.3 miles Destination will be onyour left” in navigation box 750. The mapping application also haschanged the appearance of the 3D control 150 to indicate the map is nowback in 3D.

In some embodiments, the navigation application determines some or allof the following five pieces of information for every location update(e.g., 1 time per second). First, the navigation application determinesthe location of the point of reference (i.e. the user's location).

Second, the navigation application determines the location of the pointof focus of the virtual camera, which is used to determine whichdirection the virtual camera should face. If the user is off-route, thepoint of focus will be a fixed distance ahead of the user along theuser's direction of travel (if that can be determined) or a fixeddistance north of the user (if the user's direction of travel cannot bedetermined). If the user is on-route, the point of focus will be a fixeddistance ahead of the user along the route, with the angle between thevector from the user and this point of focus and the user's traveldirection capped at a maximum value. This allows the virtual camera tosubtly peek around turns before the user actually turns. For example, ifthe route turns a corner shortly ahead, the point of focus will be apoint around the corner from the current location of the device. Asturning the virtual camera to face that actual point could cause thevirtual camera to directly face a building, the virtual camera is cappedas to how far off the present direction it can look. Third, thenavigation application determines the location of the point of interest(e.g., the location of an upcoming intersection).

Fourth, the navigation application determines the virtual camera viewstyle (top-down centered, top-down forward, or rooftop). “Top-downcentered” means that the virtual camera should look straight down on theuser's location such that the user's location is in the center of thescreen. “Top-down forward” means the virtual camera should look straightdown on the user's location such that the user's location is toward thebottom of the screen. “Rooftop” means the virtual camera should bebehind the user's location and pitched so that it is looking forwardalong the vector from the user's location to the point of focus. If theuser is off-route or the user's direction of travel cannot be determined(e.g., when the user is parked), the virtual camera will be in top-downcentered view style. Otherwise, the view style will be determined bywhether the user has requested “2D” navigation or not. If the user hasrequested 2D navigation, the view style will be top-down forward.Otherwise, the view style will be rooftop.

Fifth, the navigation application determines the virtual camera focusstyle (e.g., cruise or hard focus). “Cruise focus style” means thevirtual camera should adopt a preset height and pitch angle based on theview style. “Hard focus” means that the virtual camera should adjust itsheight (in the case of top-down centered or top-down forward viewstyles) or pitch (in the case of rooftop view style) so that the givenpoint-of-interest is just on screen (i.e. the virtual camera shouldfocus in on the point-of-interest as the user approaches it). When farfrom an intersection, the navigation application puts the virtual camerain cruise focus mode. When approaching an ‘interesting’ intersection,the navigation application puts the virtual camera in hard focus mode asdescribed above and the location of the intersection (point of interest)will be passed to the virtual camera. When in hard focus mode, theapplication adjusts the virtual camera's height (in the case of top-downcentered or top-down forward view styles) or pitch (in the case ofrooftop view style) so that the intersection is at a reasonable positionon screen. A given intersection is determined to be ‘interesting’ enoughto focus on using the angle at which the user will leave theintersection. If the angle is large enough (e.g., a 90 degree rightturn), the intersection is considered to be ‘interesting’ and thevirtual camera will focus on it. If the angle is too small (e.g.,merging onto a freeway), the virtual camera will stay in cruise focusstyle

From these five pieces of information, the navigation applicationcomputes the virtual camera's desired position and orientation. From thedesired position and orientation, the positions of the following threekey points can be extracted: (1) the virtual camera's position, (2) theintersection between the virtual camera's forward vector and the ground,and (3) a point along the virtual camera's right vector. The threepoints are animated independently from each other as follows: (1) when anew point is available, the application fits a cubic polynomial betweenthe last evaluated position/tangent for that point and the new point and(2) every step of the animation, the navigation application evaluatesthe cubic polynomials for each curve and extracts the virtual cameraposition and orientation from them.

4. User Adjustment of Camera Height

Besides (or instead of) having the navigation application control thecamera (e.g., turning from 3D to 2D when going around corners) someembodiments also allow the user to adjust the level of the camera. Someembodiments allow the user to make a command gesture with two fingers toadjust the distance (height) and angle of the camera. Some embodimentseven allow multiple types of gestures to control the camera. FIG. 8illustrates the adjustment of the distance of a virtual camera bycontracting and expanding gestures. The figure is shown in three stages801-803. In stage 801, the application shows a basic scene 810 with avirtual camera 812 at the default level for 3D viewing and the screenview 814 rendered from the scene 810. The basic scene contains twobuildings and a T-junction. In stage 801, the buildings are viewed froma 45 degree downward angle and a particular height that makes them seema particular size. The location indicator 816 is also shown at aparticular size.

In stage 802, the user makes a gesture by placing two fingertips neareach other on the screen of the device, on the screen view 824 andmoving the fingertips apart while they are on the screen. Moving thefingertips apart has the effect of making the map (both the part betweenthe fingers and the rest of the map) larger. In order to make the thingsin the map appear larger, the application causes the virtual camera 812to zoom in. In some embodiments, the line 850 that the mappingapplication uses to move the virtual camera 812 along is a line formedby the front of the virtual camera 812 and the virtual camera 812'spoint of focus. The mapping application of some embodiments moves thevirtual camera 812 along a line formed by the front of the virtualcamera 812 and a location in the 3D map 810 based on the user's input tozoom into the view of the 3D map 810.

After zooming in for stage 802, the user decides to zoom out for stage803. In this stage the user has placed two fingers on the screen andbrought them closer together. Bringing the fingers closer together hasthe effect of shrinking the map (both the part between the fingers andthe rest of the map). The zoom-out adjustment is accomplished by movingthe virtual camera 812 moving farther away from the 3D map 810 along theline 855. In some embodiments, the line 855 that the mapping applicationuses to move the virtual camera 812 along is a line formed by the frontof the virtual camera 812 and the virtual camera 812's point of focus.The mapping application of some embodiments moves the virtual camera 812along a line formed by the front of the virtual camera 812 and alocation in the 3D map 810 based on the user's input to zoom into theview of the 3D map 810.

Rendering a 3D map view using the virtual camera 812 at this positionresults in a 3D map view 834 in which the buildings and the roads appearfarther than the position illustrated in the 3D map view 824. As shownby the dashed-line version of the virtual camera 812, the virtual camera812 moved farther from the 3D map 810 along the line 855.

In addition to being controllable by zooming in and out, someapplications allow a user to change the angle of the virtual camera.FIG. 9 illustrates an embodiment of a camera whose angle can be adjustedby gestures. The figure is shown in three stages 901-903. In stage 901,the camera is pointing downward at 45 degrees at scene 910. Scene 910contains two buildings and a T-junction which are shown in screen view914. The buildings are shown from a particular angle and a particularsize. The location indicator 916 is also shown at a particular size.

In stage 902, the user has placed two fingers 920 on the screenapproximately horizontal to each other and dragged up. This has theapparent effect of dragging the scene up with the fingers. The scenerising is accomplished by the virtual camera 912 lowering and changingits viewing angle from 45 degrees to 30 degrees. In the screen view 924,the buildings and the location indicator look taller than in stage 901.

After the user drags the scene up in stage 902, the user then drags thescene down in stage 903. To do this, the user again placed two fingers930 on the screen and drags them down. This drags the scene down alongwith the fingers 930. The scene dropping is accomplished by the virtualcamera 912 rising and changing its angle with the scene 910 to 60degrees downward. In stage 903, the camera 912 has moved farther up andis angled down more than in stage 901. Accordingly, the buildings andlocation identifier 916 again look shorter and smaller in stage 903 thanin stage 901.

In some embodiments, the mapping application provides an inertia effectfor different operations (e.g., panning, rotating, entering from 2D to3D). When a user provides a particular type of input (e.g., input thatterminates at a velocity greater than a threshold velocity) to pan the3D map, the mapping application generates an inertia effect that causesthe 3D map to continue panning and decelerate to a stop. The inertiaeffect in some embodiments provides the user with a more realisticinteraction with the 3D map that mimics behaviors in the real world.Details of inertia effects and implementations of inertia effects aredescribed in U.S. patent application Ser. No. 13/632,040, filed Sep. 30,2012, entitled “Virtual Camera for 3D Maps,” which is incorporatedherein by reference.

The application of some embodiments allows the distance and angle of thecamera to be independently controlled. For example, it allows thedistance to be controlled by the contracting and expanding fingergestures and the angle to be controlled by the dragging of horizontallyplaced fingers. Other embodiments use whichever gesture is beingperformed to set either a distance or an angle of the camera, with theother variable being set automatically. While FIGS. 8 and 9 showgestures being performed in a certain direction leading to certainresults, in some embodiments, one or both of these gestures could bereversed. For example, in some embodiments, dragging horizontally placedfingers down may bring the camera down rather than bringing the scenedown. That would have the effect of moving the scene down when thefingers move up and moving the scene up when the fingers move down.

FIG. 10 conceptually illustrates a feature provided by the mappingapplication of some embodiments for maintaining the position of avirtual camera within a defined range along an arc. In particular, FIG.10 illustrates the virtual camera 1000 at three different stages1005-1015 that show the virtual camera 1000's position maintained withina defined range of arc 1050. As shown in FIG. 10, a location in a 3D map1035 includes two buildings and two roads forming a T-junction.

The first stage 1005 shows the virtual camera 1000 at a particularposition along the arc 1050. As shown, the arc 1050 represents a definedrange (e.g., angular range) within which the virtual camera 1000 ismovable. The first stage 1005 also shows three positions 1055-1065 alongthe arc 1050 (e.g., perspective view angles). In this example, themapping application moves the virtual camera 1000 along the arc 1050between the high perspective end of the arc 1050 (e.g., the positionalong the arc 1050 when the virtual camera 1000 is most tilteddownwards) and the position 1055 in a manner similar to that describedabove by reference to FIG. 9. Rendering a 3D map view based on thevirtual camera 1000's position in the first stage 1005 results in 3D mapview 1025.

When the virtual camera 1000 passes the position 1055 while movingtowards the low perspective end of the arc 1050, the mapping applicationreduces the speed (e.g., decelerates) that the virtual camera 1000 movestowards the low perspective end of the arc 1050 regardless of the inputprovided by a user. In some embodiments, the mapping application reducesthe speed of the virtual camera 1000 at a constant rate while, in otherembodiments, the mapping application reduces the speed of the virtualcamera 1000 at an exponential rate. Additional and/or different methodsfor decreasing the speed of the virtual camera 1000 are used in someembodiments.

The second stage 1010 shows that the virtual camera 1000 has moved to aposition along the arc 1050 at or near the low perspective end of thearc 1050. As shown, a user is providing input to adjust the perspectiveof the view of the 3D map 1035 by touching two fingers on the screen anddragging the two fingers in an upward direction (e.g., a swipe gesture).In response to the input, the mapping application moved the virtualcamera 1000 toward the low perspective end of the arc 1050 while tiltingthe virtual camera 1050 upwards. When the virtual camera reaches theposition 1065 along the arc 1050, the mapping application prevents thevirtual camera 1000 from moving lower and beyond the position 1065 evenwhile the user continues to provide input to decrease the perspective ofthe view of the 3D map 1035 (e.g., the user continues to drag the twofingers upwards on the touchscreen).

In some embodiments, when the user stops providing input to decrease theperspective of the view of the 3D map 1035 (e.g., the user lifts the twofingers off the touchscreen), the mapping application “bounces” or“snaps” the position of the virtual camera 1000 from the position 1065up to the position 1060 along the arc 1050. As the mapping applicationis generating or rendering 3D map views of the 3D map 1035 based on theview of the virtual camera 1000 during the bounce or snap motion, thegenerated 3D map views provide a bounce animation that displays the 3Dmap view briefly bouncing or snapping down in order to indicate to theuser that the perspective of the map view cannot be decreased anyfarther. Rendering a 3D map view using the virtual camera 1000positioned at this angle results in a 3D map view 1030 in which thebuildings and the roads are taller compared to the map view 1025.

The third stage 1015 shows the virtual camera 1000 after the mappingapplication has bounced or snapped the position of the virtual camera1000 to the position 1060 in response to the user ceasing to provideinput. Different embodiments use different techniques for implementingthe bounce or snap of the virtual camera 1000. For instance, the mappingapplication of some embodiments starts quickly accelerating the virtualcamera 1000 along the arc 1050 for a defined distance or until thevirtual camera 1000 reaches a defined speed. Then the mappingapplication decelerates the virtual camera 1000 for the remainingdistance to the position 1060 along the arc 1050. Other ways toimplement the bounce or snap effect are used in some embodiments.Rendering a 3D map view using the virtual camera 1000 positioned at theposition 1060 along the arc 1050 in the third stage 1015 results in a 3Dmap view 1040 in which the buildings appear a little smaller and flatterand the roads appear a little smaller compared to the map view 1030.

As described above, FIG. 10 illustrates a technique for preventing avirtual camera from moving beyond the low perspective end of an arc.Alternatively or in conjunction with preventing the virtual camera frommoving beyond the low perspective end of the arc, the mappingapplication of some embodiments utilizes a similar technique forpreventing the virtual camera from moving beyond the high perspectiveend of the arc. In addition, FIG. 10 shows an example of a positionalong an arc at which to slow down a virtual camera, a position alongthe arc to prevent the virtual camera from moving past, and a positionalong the arc to which the virtual camera snaps or bounces back.Different embodiments define the positions any number of different ways.For instance, in some embodiments, the position along the arc at whichto slow down the virtual camera is the same or near the position alongthe arc to which the virtual camera snaps or bounces back.

C. Other User Interactions

1. Appearing and Disappearing Controls

The applications of some embodiments, while navigating, have a fullscreen mode. That is, during the actual providing of directions, thecontrols that ordinarily take up some of the screen surface are hidden.FIG. 11 illustrates a full screen mode of some embodiments. The figureis shown in six stages 1101-1106. In stage 1101 a set of navigationinstructions is activated by the selection of a start button 1110. Byselecting the start button, the user selects the highlighted route fromtwo possible routes. The non-highlighted route disappears, and a smallerscale navigation map 1121 appears in stage 1102. The first stage 1101shows that the road names are on the roads because the mappingapplication is displaying a map view. The first stage 1101 also showsthat the position control 1130 is displayed for the mapping applicationis displaying a map view. The selection of the list control 1132 willcause the mapping application to display the available routes in a listformat.

Also in stage 1102, the first instruction 1120 is shown along with anend control 1122, trip status area 1124 (including an ETA, a tripduration estimate, and a distance of planned route indicator), anoverview button 1126, status bar 1127, and a 3D control 1128. The endbutton 1122 ends the running of the navigation instructions. The statusarea 1124 displays information about the planned route. The overviewbutton 1126 displays an overview of the route. The 3D control is anindicator of whether the navigation application is showing a scene in 3Dor 2D and a toggle for entering and leaving 3D mode. The selection ofthe list control 1132 at this stage will cause the mapping applicationto display the set of navigation instructions in a list format. Thisstage also shows that the road names are displayed in banners ratherthan on the roads because the mapping application is operating in thenavigation mode.

After a brief amount of time, the end control 1122, the list control1132, status area 1124, overview button 1126, and 3D control 1128disappear. In some embodiments, the controls disappear abruptly, whilein other embodiments the controls fade away. In some embodiments, thestatus bar 1127 at the top of the screen also vanishes and navigationbox 1120 moves to the top of the screen.

The absence of the controls and movement of navigation box 1120 is shownin stage 1103, in which the navigation map 1121 is seen without thecontrols except for the raised navigation box 1120. The user can restorethe hidden controls by tapping the screen in some embodiments. This isdemonstrated in stages 1104 and 1105. In stage 1104, the user taps thescreen with finger 1140. In stage 1105, as a result of the tap in theprevious stage, the controls are back and the navigation box 1120 hasdropped back down to its original position. The restored controlsinclude end control 1122, status area 1124, overview button 1126, statusbar 1127, and 3D control 1128. Once the controls are back, the user canmake the controls vanish again by tapping, as shown in stage 1105 wherea user taps the screen with finger 1150 to restore the navigationapplication to full screen mode in stage 1106. In addition to the hiddencontrols, in full-screen in some embodiments the touch interaction withthe map is greatly restricted. In some embodiments, more controls existthat are shown in some modes, but hidden in full screen mode (e.g., alist control).

In some embodiments, when the controls are shown and there is anaddition to the status bar (e.g., a phone call status bar showing thelength of an ongoing call) the navigation box is shortened in order tomake more room for the expanded status bar. This is shown in FIG. 12,which illustrates the navigation application with the controls hiddenand revealed during a phone call on the device. FIG. 12 includes stages1201 and 1202. In stage 1201 the controls of the navigation applicationare hidden and the navigation box 1210 and map 1215 are visible. Theuser taps on the touchscreen with finger 1217 to command the navigationapplication to show its controls. In stage 1202, the navigationapplication shows its controls 1220 and also shows a phone call statusbar 1222 under the status bar 1224. The navigation application has lessroom due to the phone call status bar 1222. To compensate for thesmaller amount of screen area available to the navigation application,the navigation application of some embodiments shrinks the navigationbox 1210 when the phone call status bar 1222 is on the screen. In someembodiments, when the navigation box shrinks, the text and/or directionarrow in the box is altered to fit the reduced amount of area availablefor the text and arrow.

2. Ending Navigation

In the ordinary course of the running of a set of navigationinstructions by a navigation application, as the device reaches each newjunction that needs navigation instructions, the instructions for thenext such junction appear. This continues until the device reaches itsdestination. When the destination is reached, the navigation applicationstops providing instructions and the running of the programmed routeends. FIG. 13 illustrates in four stages 1301-1304 the end of aprogrammed route. In stage 1301, the application is running with hiddencontrols and the navigation box 1310 is showing that the destination isonly 1000 feet away. The destination is shown on the map as a pin 1312with a round head. However, one of ordinary skill in the art willunderstand that other symbols could be used in applications of otherembodiments and that in some embodiments no symbol is used and the linemerely ends. As the device moves closer to its destination, thenavigation application counts down the distance. In stage 1302, thenavigation box 1320 shows that there are only 100 feet to go to thedestination. In stage 1303, the device has just reached its destination.Navigation box 1330 indicates that the destination is on the left andincludes a symbol of an arrow pointing at the center of a target. Later,in stage 1304, with the device having reached its destination, thenavigation application has shut navigation box 1320 down leaving theuser with a map 1340, but no further directions.

In some embodiments, destinations can be in places not reachable by car,for example, the end pin could be in the middle of a park. In some suchembodiments, the driving directions will end, but there will becontinued directions for foot travel. In other such embodiments, theapplication will not give textual directions for travel on foot, butwill still maintain a pin on the location (e.g., the middle of a park)when displaying maps in map mode or in locked mode. In some suchembodiments, the last instruction after the automotive portion of thejourney ends will be a direction “please reach on foot”.

FIG. 13 illustrates what happens when a navigation application guidesthe user all the way to its final destination. However, in someembodiments, the user may change the user's mind about gettingdirections. The user may want to stop along the way, changedestinations, or for some other reason, may want to end the running ofthe set of navigation instructions. Accordingly, the application of someembodiments includes an “end” button. The end button stops the runningof a set of navigation instructions and in some embodiments leaves theuser in the same condition as if they had reached the destination (e.g.,no instructions but with a map). FIG. 14 illustrates a navigationprogram ending control. The figure is shown in two stages 1401 and 1402.Stage 1401 shows a navigation application with its controls visible. Thecontrols include an “end” button 1410. The user is tapping the buttonwith finger 1412. The navigation application is far from itsdestination, as indicated by navigation box 1414, which states that thenext junction is 20 miles away, and by route 1416, which stretches offinto the distance ahead of position indicator 1418. In stage 1402,because the user has tapped the end button 1410, the navigation box 1414disappears as does the route 1416. The position indicator 1418 is alsogone in this stage, replaced by a spherical position indicator 1428.

3. Gestures to Look to the Side of the Route During Navigation

As described above, the default behavior for the virtual camera is tofollow the location of the device through a virtual world and point downand in the direction the device is moving, or at least to a part of itsroute a short way ahead of the device's present position. However, it isnot always desirable to have the camera pointing straight ahead.Sometimes the user wants the camera to point at an angle instead.Accordingly, the navigation application of some embodiments rotates thevirtual camera around when the user drags the map sideways.

FIG. 15 illustrates the rotation of a map when a user pushes itsideways. The figure includes four stages 1501-1504. In stage 1501, theapplication is shown in its default mode, with the street 1510 (MainSt.) and the current route 1512 running parallel to the sides of thescreen on the 3D map 1514. In this stage 1501 the user begins pushingthe map to the left. In the next stage 1502, the virtual camera hasmoved to the left and rotated to the right. That is, the 3D map 1514 haschanged as though the virtual camera has moved to the left and rotatedto the right. The map 1514, having been rotated, now shows the faces ofthe buildings on the right side of the street. In some embodiments,there is a maximum threshold to how far the map will rotate. In someembodiments, as well as being able to move the map from side to side,the user can move to a view slightly ahead of or slightly behind thelocation indicator (e.g., by dragging down or up with one finger). Insome such embodiments, the amount that the map can be moved ahead orbehind by dragging is also capped.

In the illustrated embodiment, the application only rotates thebuildings while the user is dragging the map to the left (or right), orfor a short time after (e.g., with simulated inertia). Once the userstops dragging the map 1514 or holding his finger in place to hold themap 1514 in place, the map 1514 reverts to its default view in thedirection of the route the camera is taking. This is shown in stage 1503in which the user has stopped dragging the map 1514 and the virtualcamera is rotating and/or moving back to its original position directlybehind the device as it moves on its route. By stage 1504, the map 1514has resumed its previous orientation. In some embodiments, the virtualcamera merely rotates when the map is dragged sideways, rather thanmoving as well as rotating. While in other embodiments, the camerarevolves around the location identifier so that the location identifierappears to be a fixed point while the map revolves around it.

4. Route Overview Mode

In some cases, rather than looking at only a small scale map that showsthe next junction, some users may sometimes want to get a look at thebig picture. That is, the users may want to look at the entirety oftheir navigation application's planned route while the user is travelingover the route. Therefore some embodiments provide an overview optionthat shows the user the entire route. FIGS. 16 and 17 illustrateoverview controls. FIG. 16 includes two stages 1601 and 1602. In stage1601 a navigation map 1610, overview button 1612, finger 1614, and listcontrol 1617 are shown. In navigation map 1610, the location indicator1616, shows that the device is on Main St. close to 1st St. The stage1601 also shows that the mapping application is displaying the roadnames in banners 1618 because the mapping application is operating inthe navigation mode. In this stage the finger 1614 hits overview button1612 causing the overview to be displayed in stage 1602.

In stage 1602, the navigation application has displayed an overview map1620, resume button 1622, location indicator pin 1626, end pin 1628 andposition indicator control 1630. The overview map 1620 shows the userhis entire planned route starting from the present position. In theillustrated embodiment, the overview map focuses on the remaining route,not the entire route from the beginning, as it does not show a lightcolored line indicating the previously traveled route. However, in someembodiments, the overview map shows the entire route rather than justthe route from the current location of the device. In some embodiments,list control 1617 is also present in the overview map to allow the userto go directly from the overview map to a list of maneuvers (e.g.,upcoming turns). The second stage 1602 also shows that the road namesare displayed on the road because the mapping application is displayingthe overview map (i.e., not in the navigation mode). It is to be notedthat the mapping application of some embodiments alternatively orconjunctively uses banners to display the road names regardless of themode in which the mapping application is operating.

The resume button 1622 switches the navigation application back to thenavigation view of stage 1601. The location indicator pin 1626 and theend pin 1628 show the current location of the device and the finaldestination of the navigation route, respectively. In some embodiments,the application allows a user to move the map around, zoom in and out,and otherwise focus on different parts of the overview map 1620. Theposition indicator control 1630 in some embodiments centers the map onthe location indicator pin 1626.

In some embodiments, the overview mode has a search box that allows auser to enter search queries for items that may be found in the overviewmap. For example, the user could search for gas stations on the map sothat the user can determine where to refuel his car. Another examplewould be a search for coffee shops so the user could stop for coffee.Some embodiments allow a user to switch from an original end destinationto a destination found in a search before resuming navigation.

In some embodiments all overview maps are 2D. In other embodiments, someor all overview maps are in 3D. For example, some embodiments use 2Doverview maps for routes that cover large distances, but use 3D overviewmaps for navigation routes that cover short distances. FIG. 17illustrates an embodiment that uses 3D overview maps. FIG. 17 includestwo stages 1701 and 1702. In stage 1701 a navigation map 1710, overviewbutton 1712, finger 1714, and list button 1617 are shown. In navigationmap 1710, the location indicator 1716 shows that the device is on MainSt. close to 1st St. In this stage the finger 1714 hits overview button1712 causing the overview to be displayed in stage 1702.

In stage 1702, the navigation application has displayed an overview map1720, resume button 1722, location indicator pin 1726, end pin 1728 andposition indicator control 1730. The overview map 1720 shows the usertheir entire planned route. The resume button 1722 switches thenavigation application back to the navigation view of stage 1701. Thelocation indicator pin 1726 and end pin 1728 show the current locationof the device and the final destination of the navigation route,respectively. The position indicator control 1730 centers the map on thelocation indicator pin 1726.

In some embodiments, the 3D overview maps include a search function asdescribed with respect to FIG. 16. Also, in some embodiments, theoverview mode includes a control to center the map on the end pin. Insome embodiments, the position indicator control allows a user to togglebetween centering on the present location of the device and thedestination of the device. In some embodiments, the overview mode can beactivated at any time while navigating.

D. Multi-Mode Application

1. Rendering Module

FIG. 18 conceptually illustrates a processing, or map rendering,pipeline 1800 performed by the mapping application of some embodimentsin order to render a map for display at the client device (e.g., on thedisplay of the client device). In some embodiments, the map renderingpipeline 1800 may be referred to collectively as a map rendering module.A more detailed version of this processing pipeline is described in U.S.patent application Ser. No. 13/632,040, filed Sep. 30, 212, entitled“Virtual Camera for 3D Maps,” which is incorporated herein by reference.As illustrated, the processing pipeline 1800 includes tile retrievers1805, a set of mesh builders 1815, a set of mesh building processors1810, a tile provider 1820, a virtual camera 1830, and a map renderingengine 1825.

The tile retrievers 1805 perform various processes to retrieve map tilesin some embodiments, according to requests for the map tiles from themesh builders 1815. The mesh builders 1815, as will be described below,identify existing map tiles (that are stored on a mapping service serveror in a cache on the device performing the processing pipeline 1800)needed to build their respective meshes. The tile retrievers 1805receive the requests for the map tiles, determine the best location fromwhich to retrieve the map tiles (e.g., from the mapping service, from acache on the device) and decompress the map tiles if required.

The mesh builders 1815 (also referred to as tile sources) of someembodiments are instantiated by the tile provider 1820 in order to builddifferent layers of view tiles. Depending on the type of map beingdisplayed by the mapping application, the tile provider 1820 mayinstantiate a different number and different types of mesh builders1815. For instance, for a flyover (or satellite) view map, the tileprovider 1820 might only instantiate one mesh builder 1815, as theflyover map tiles of some embodiments do not contain multiple layers ofdata. In fact, in some embodiments, the flyover map tiles contain analready-built mesh generated at the mapping service for which theflyover images (taken by a satellite, airplane, helicopter, etc.) areused as textures. However, in some embodiments, additional mesh buildersmay be instantiated for generating the labels to overlay on the flyoverimages when the application is in a hybrid mode. For a 2D or 3D renderedvector map (i.e., a non-satellite image map), some embodimentsinstantiate separate mesh builders 1815 to build meshes for landcoverpolygon data (e.g., parks, bodies of water, etc.), roads, place ofinterest markers, point labels (e.g., labels for parks, etc.), roadlabels, traffic (if displaying traffic), buildings, raster data (forcertain objects at certain zoom levels), as well as other layers of datato incorporate into the map.

The mesh builders 1815 of some embodiments, receive “empty” view tilesfrom the tile provider 1820 and return “built” view tiles to the tileprovider 1820. That is, the tile provider 1820 sends to each of the meshbuilders 1815 one or more view tiles (not shown). Each of the view tilesindicates an area of the world for which to draw a mesh. Upon receivingsuch a view tile, a mesh builder 1815 identifies the map tiles neededfrom the mapping service, and sends its list to the tile retrievers1805.

Upon receiving the tiles back from the tile retrievers 1805, the meshbuilder uses vector data stored in the tiles to build a polygon mesh forthe area described by the view tile. In some embodiments, the meshbuilder 1815 uses several different mesh building processors 1810 tobuild the mesh. These functions may include a mesh generator, atriangulator, a shadow generator, and/or a texture decoder. In someembodiments, these functions (and additional mesh building functions)are available to each mesh builder, with different mesh builders 1815using different functions. After building its mesh, each mesh builder1815 returns its view tiles to the tile provider 1820 with its layer ofthe mesh filled in.

The tile provider 1820 receives from the controller 1875 a particularview (i.e., a volume, or viewing frustrum) that represents the map viewto be displayed (i.e., the volume visible from the virtual camera 1830).The tile provider performs any culling (e.g., identifying the surfacearea to be displayed in the view tile), then sends these view tiles tothe mesh builders 1815.

The tile provider 1820 then receives the built view tiles from the meshbuilders and, in some embodiments, performs culling on the built meshusing the particular view from the virtual camera 1830 (e.g., removingsurface area too far away, removing objects that will be entirely behindother objects, etc.). In some embodiments, the tile provider 1820receives the built view tiles from the different mesh builders atdifferent times (e.g., due to different processing times to completemore and less complicated meshes, different time elapsed beforereceiving the necessary map tiles from the tile retrievers 1805, etc.).Once all of the layers of view tiles have been returned, the tileprovider 1820 of some embodiments puts the layers together and releasesthe data to the controller 1875 for rendering.

The virtual camera 1830 generates a volume or surface for the pipeline1800 to render, and sends this information to the controller 1875. Basedon a particular location and orientation from which the map will berendered (i.e., the point in 3D space from which the user “views” themap), the virtual camera identifies a field of view to actually send tothe tile provider 1820. In some embodiments, when the mappingapplication is rendering the 3D perspective view for navigation, thefield of view of the virtual camera is determined according to analgorithm that generates a new virtual camera location and orientationat regular intervals based on the movement of the user device.

The controller 1875 is responsible for managing the tile provider 1820,virtual camera 1830, and map rendering engine 1825 in some embodiments.In some embodiments, multiple tile providers may actually beinstantiated, and the controller puts together several view tiles (e.g.,map tiles and building tiles) to create a scene that is handed off tothe map rendering engine 1825.

The map rendering engine 1825 is responsible for generating a drawing tooutput to a display device based on the mesh tiles (not shown) sent fromthe virtual camera. The map rendering engine 1825 of some embodimentshas several sub-processes. In some embodiments, each different type ofmap element is rendered by a different sub-process, with the renderingengine 1825 handling the occlusion of different layers of objects (e.g.,placing labels above or behind different buildings, generating roads ontop of land cover, etc.). Examples of such rendering processes include aroad rendering process, a building rendering process, a label renderingprocess, a vegetation rendering process, a raster traffic renderingprocess, a raster road rendering process, a satellite rendering process,a polygon rendering process, a background raster rendering process, etc.

The operation of the rendering pipeline 1800 in some embodiments willnow be described. Based on user input to view a particular map region ata particular zoom level, the virtual camera 1830 specifies a locationand orientation from which to view the map region, and sends thisviewing frustrum, or volume, to the controller 1875. The controller 1875instantiates one or more tile providers. While one tile provider 1820 isshown in this figure, some embodiments allow the instantiation ofmultiple tile providers at once. For instance, some embodimentsinstantiate separate tile providers for building tiles and for maptiles.

The tile provider 1820 performs any culling necessary to generate anempty view tile identifying regions of the map for which a mesh needs tobe built, and sends the empty view tile to the mesh builders 1815, whichare instantiated for the different layers of the drawn map (e.g., roads,land cover, POI labels, etc.). The mesh builders 1815 use a manifestreceived from the mapping service that identifies the different tilesavailable on the mapping service server (i.e., as nodes of a quadtree).The mesh builders 1815 request specific map tiles from the tileretrievers 1805, which return the requested map tiles to the meshbuilders 1815.

Once a particular mesh builder 1815 has received its map tiles, itbegins using the vector data stored in the map tiles to build the meshfor the view tiles sent from the tile provider 1820. After building themesh for its map layer, the mesh builder 1815 sends the built view tileback to the tile provider 1820. The tile provider 1820 waits until ithas received all of the view tiles from the various mesh builders 1815,then layers these together and sends the completed view tile to thecontroller 1875. The controller stitches together the returned tilesfrom all of its tile providers (e.g., a map view tile and a buildingview tile) and sends this scene to the rendering engine 1825. The maprendering engine 1825 uses the information in the map tiles to draw thescene for display.

2. State Diagram for Different Modes

FIG. 19 conceptually illustrates a state diagram 1900 that describesdifferent states and transitions between these states of the integratedmapping, search, and navigation application of some embodiments (e.g.,the application described in the above sections). One of ordinary skillin the art will recognize that the application of some embodiments willhave many different states relating to all different types of inputevents, and that the state diagram 1900 is specifically focused on asubset of these events. The state diagram 1900 describes and refers tovarious gestural interactions (e.g., multi-touch gestures) for changingstates of the application. One of ordinary skill in the art willrecognize that various other interactions, such as cursor controllergestures and button clicks, keyboard input, touchpad/trackpad input,etc., may also be used for similar selection operations.

When a user initially opens the mapping application, the application isin state 1905, the map browsing state. In this state 1905, theapplication will have generated and displayed a map view. To generateand display this map view, the application of some embodimentsidentifies a required set of map tiles for a region, requests the maptiles (e.g., from a mapping service server), generates a view of the maptiles from a particular location, orientation, and perspective of avirtual camera, and renders the map view to a device display. When instate 1905, the map view is static. With the application in state 1905,the user can perform numerous operations to modify the map view, searchfor entities (e.g., places of interest, addresses, etc.), retrieve aroute for navigation, etc.

In some embodiments, the integrated application is displayed on a devicewith an integrated touch-sensitive display. Various gesturalinteractions over the map may cause the application to perform differentmodifications to the map view (e.g., panning, rotating, zooming,modifying the map perspective, etc.). When the integrated applicationreceives gestural interactions over the map display (as opposed to touchinputs over various floating or non-floating controls overlaid on themap display), the application transitions to state 1910 to performgestural input recognition.

The gestural input recognition state 1910 differentiates betweendifferent types of gestural input and translates these types of inputinto different map view modification operations. In some embodiments,the mapping application receives the gestural input as translated by theoperating system of the device with the integrated touch-sensitivedisplay. The operating system translates the touch input into gesturetypes and locations (e.g., a “tap” at coordinates (x,y), a “pinch”operation with separate touch inputs at two different locations, etc.).At state 1910, the integrated mapping application of some embodimentstranslates these into the different map view modification operations.

When the application receives a first type of gestural input (e.g., twoseparate touch inputs moving together in a rotational motion over themap view), the application transitions to state 1915 to rotate the map.To rotate the map view, some embodiments modify the location and/ororientation of the virtual camera that determines which portion of themap is rendered to create the map view. When in 3D mode, for example,the mapping application rotates the virtual camera about a particularposition (e.g., the center of the touch inputs, the center of thedisplay, a location indicator identifying the user's location, etc.). Asthe first type of gestural input continues, the mapping applicationremains in state 1915 to continue rotating the map.

When the user releases the first type of gestural input, the applicationof some embodiments transitions to state 1930 to perform an inertiacalculation. In some embodiments, after the user releases certain typesof touch inputs, the application continues to perform the associated mapview modification for a particular amount of time and/or distance. Inthis case, after the user releases the rotation input, the applicationtransitions to the inertia calculation state 1930 to calculate theadditional rotation amount and the time over which this rotation shouldbe performed. In some embodiments, the application slows down therotation from the (angular) velocity at which the map was being rotated,as if a “frictional” force was applied to the map. As such, the inertiacalculation of some embodiments is based on the speed of the first typeof gestural input. From state 1930, the application transitions back tothe map modification state that the application was previously in. Thatis, when the application transitions from state 1915 (the rotationstate) to the inertia calculation state 1930, it then transitions backto state 1915 after performing the inertia calculation. After therotation of the map is complete, the application transitions back tostate 1905.

When the application receives a second type of gestural input (e.g., asingle touch input moving over the map view), the applicationtransitions to state 1920 to pan the map. To pan the map view, someembodiments modify the location of the virtual camera that determineswhich portion of the map is rendered to create the map view. This causesthe map to appear to slide in a direction derived from the direction ofthe second type of gestural input. In some embodiments, when the mapview is in a 3D perspective mode, the panning process involvesperforming a correlation of the location of the touch input to alocation on the flat map, in order to avoid sudden unwanted jumps in themap view. As the second type of gestural input continues, the mappingapplication remains in state 1920 to continue panning the map.

When the user releases the second type of gestural input, theapplication of some embodiments transitions to state 1930 to perform aninertia calculation. In some embodiments, after the user releasescertain types of touch inputs, the application continues to perform theassociated map view modification for a particular amount of time and/ordistance. In this case, after the user releases the panning input, theapplication transitions to the inertia calculation state 1930 tocalculate the additional amount to move the map view (i.e., move thevirtual camera) and the time over which this movement should beperformed. In some embodiments, the application slows down the panningmovement from the velocity at which the map was being panned, as if a“frictional” force was applied to the map. As such, the inertiacalculation of some embodiments is based on the speed of the second typeof gestural input. From state 1930, the application transitions back tothe map modification state that the application was previously in. Thatis, when the application transitions from state 1920 (the panning state)to the inertia calculation state 1930, it then transitions back to state1920 after performing the inertia calculation. After the panning of themap is complete, the application transitions back to state 1905.

When the application receives a third type of gestural input (e.g., twoseparate touch inputs moving closer together or farther apart), theapplication transitions to state 1925 to zoom in on or out of the map.To change the zoom level of the map view, some embodiments modify thelocation (i.e., height) of the virtual camera that determines whichportion of the map is rendered to create the map view. This causes themap view to include more (if zooming out) or less (if zooming in) of themap. In some embodiments, as the user zooms in or out, the applicationretrieves different map tiles (for different zoom levels) to generateand render the new map view. As the third type of gestural inputcontinues, the mapping application remains in state 1925 to continuezooming in on or out of the map.

When the user releases the second type of gestural input, theapplication of some embodiments transitions to state 1930 to perform aninertia calculation. In some embodiments, after the user releasescertain types of touch inputs, the application continues to perform theassociated map view modification for a particular amount of time and/ordistance (i.e., moving the virtual camera higher or lower). In thiscase, after the user releases the zoom input, the applicationtransitions to the inertia calculation state 1930 to calculate theadditional amount to zoom the map view (i.e., move the virtual camera)and the time over which this movement should be performed. In someembodiments, the application slows down the zooming movement from thevelocity at which the map was being zoomed in on or out of (i.e., thespeed at which the virtual camera changes height), as if a “frictional”force was applied to the camera. As such, the inertia calculation ofsome embodiments is based on the speed of the third type of gesturalinput. From state 1930, the application transitions back to the mapmodification state that the application was previously in. That is, whenthe application transitions from state 1925 (the zooming state) to theinertia calculation state 1930, it then transitions back to state 1925after performing the inertia calculation. After the zooming of the mapis complete, the application transitions back to state 1905.

For simplicity, the state diagram 1900 illustrates the map panning,zooming, and rotation processes using the same inertia calculationprocess (state 1930). However, in some embodiments, each of thesedifferent map modification processes actually uses a different inertiacalculation to identify the slow-down and stop for its particular typeof movement. In addition, some embodiments calculate and modify theinertia variables as the input is received rather than when the userremoves the gestural input.

When the application receives a fourth type of gestural input (e.g., twoseparate touch inputs moving up or down the touch-sensitive display inunison), the application transitions to state 1935 to modify theperspective view of the map. To change the perspective view of the map,some embodiments move the virtual camera along an arc over the map,modifying both the location and orientation of the virtual camera (asthe camera keeps the center of its field of view at a particularlocation on the map). In some embodiments, different zoom levels usedifferent arcs along which the virtual camera moves. Each of these arcshas a top point at which the virtual camera is pointing straight down,giving a 2D perspective view of the map. In addition, each arc has abottom point, that is the lowest point on the arc to which the virtualcamera can be moved. Thus, the fourth type of gestural input can causethe application to change between a 2D map view and a 3D perspective mapview in some embodiments. As the fourth type of gestural inputcontinues, the mapping application remains in state 1935 to continuemodifying the perspective view of the map.

When the user releases the fourth type of gestural input, theapplication of some embodiments transitions to state 1940 to perform aninertia calculation. In some embodiments, after the user releasescertain types of touch inputs, the application continues to perform theassociated map view modification for a particular amount of time and/ordistance (i.e., moving the virtual camera higher or lower). In thiscase, after the user releases the perspective view change input, theapplication transitions to the inertia calculation state 1940 tocalculate the additional amount to modify the perspective of the mapview (i.e., move the virtual camera along its arc) and the time overwhich this movement should be performed. In some embodiments, theapplication slows down the movement from the velocity at which the mapwas changing perspective (i.e., the speed at which the virtual cameramoves along its arc), as if a “frictional” force was applied to thecamera. As such, the inertia calculation of some embodiments is based onthe speed with which the fourth type of gestural input was performed.

In addition, for the perspective change operation, some embodimentstransition to a rebound calculation state 1945. As stated, theperspective change operation has a maximum and minimum perspective shiftallowed in some embodiments, which may depend on the zoom level of thecurrent map view. Thus, in addition to an inertia calculation, theapplication performs a rebound calculation at state 1945. The reboundcalculation uses the inertia calculation to determine whether themaximum point along the virtual camera arc will be reached and, if so,the velocity of the virtual camera at this point. Some embodiments allowthe virtual camera to move slightly past the maximum point to hit a“rebound” point, at which point the application turns the virtual cameraaround on its arc, moving it back towards the maximum point. Someembodiments include such a bounce-back functionality only on one end ofthe virtual camera arc (e.g., the bottom of the arc), while otherembodiments include the functionality on both ends of the arc. From therebound calculation state 1945, the application transitions back to theinertia calculation state 1940, then back to the perspective changingstate 1935 to display the map view movement. In addition, when the userperforms the fourth type of touch input for long enough and theperspective reaches its maximum point, the application transitionsdirectly from the state 1935 to state 1945 to calculate the reboundinformation and then transitions back to state 1935. After themodification to the perspective view of the map is complete, theapplication transitions back to state 1905.

The above states relate to the various multi-touch gestures over the mappresentation that the integrated mapping, search, and navigationapplication translates into different modifications to the mappresentation. Various other touch inputs can also cause the applicationto change states and perform various functions. For instance, someembodiments overlay a 3D selectable item on the map view (e.g., as afloating control), and selecting (e.g., with a tap input) the 3D itemcauses the application to transition to 1935 to modify the perspectiveof the map view. When the map view starts in a 3D perspective view, theapplication modifies the perspective into a 2D view; when the map viewstarts in the 2D view, the application modifies the perspective into a3D view. After the modification, the application returns to state 1905.

When a user is viewing a map in state 1905, the application presentsvarious labels as part of the map view. Some of these labels indicateplaces of interest, or other locations. When a user selects certainlabels (e.g., for certain businesses, parks, etc.), the applicationtransitions to state 1950 to display a banner for the selected location(e.g., an information display banner), then returns to the map browsingstate (with the banner displayed over the map). In some embodiments,this banner includes (1) a quick-route navigation UI control (e.g., abutton) that causes the application to retrieve a route (e.g., a drivingroute) from a current location of the device to the selected locationwithout leaving the map view and (2) an information UI control (e.g.,button) that causes the application to provide additional informationabout the location.

When a user selects the UI control button, the application transitionsfrom state 1905 to state 1955 to display a staging area for the selectedlocation. In some embodiments, this staging area displays a mediapresentation of the selected location (e.g., a 3D video presentation, aflyover view of the selected location, a series of images captured forthe location, etc.), as well as various information for the selectedlocation (contact information, reviews, etc.). The application stays inthe state 1955 as the user performs various operations to navigate thestaging area and view information within the staging area. When a userselects a UI control to transfer back to the map view, the applicationtransitions to state 1905.

From the map browsing view, the user can also easily access the searchfunction of the application. When a particular UI control (e.g., asearch bar) is selected, the application transitions to a search entrysuggestion state 1960. At the search entry state, some embodimentsdisplay a touchscreen keyboard with which the user can enter a searchterm. The search term may be a business name, an address, a type oflocation (e.g., coffee shops), etc. While the user enters characters,the application remains in state 1960 and provides suggestions based onrecent searches, the letters already entered, etc. Some embodiments mayuse prefix-based suggestions (e.g., suggestions starting with thecharacters already entered) as well as other suggestions (e.g., makingspelling corrections to add characters at the beginning of thealready-entered string, transpose characters, etc.). In someembodiments, the selections may also include recently entered routes inaddition to locations. If the user selects a cancellation UI control atthis stage, the application transfers back to state 1905 withoutperforming a search.

When the user selects a search term (either a suggested term or a termentered completely by the user), the application transitions to state1965 to display the search results over the map view, then transitionsto state 1905 with the search results displayed. Some embodimentsdisplay the search results as selectable items (e.g., pins) on the map;selection of one of the items causes a transition to state 1950 todisplay the banner for the selected item. In addition, the applicationof some embodiments automatically selects one of the search results(e.g., a “best” result) and displays this banner as part of the state1965.

As the application is a tightly integrated mapping, search, routing, andnavigation application, the user can easily access the routing functionfrom the map browsing state. When a particular UI control (e.g., a routeentry button) is selected, the application transitions to the routeentry state 1970. At the route entry state, some embodiments display atouchscreen keyboard with which the user can enter locations (e.g.,addresses, place names, place types, etc.) into both “to” and “from”fields in order to request a route. While the user enters characters,the application remains in state 1970 and provides suggestions based onrecent routes, recent searches, an autocomplete similar to thatdescribed for the search entry, etc. If the user selects a cancellationUI control at this stage, the application transfers back to state 1905without retrieving a route.

When the user selects a route (e.g., by entering a “to” location and a“from” location), the application transitions to the route displayingstate 1975. At this state, the application displays one or more routesfrom a first selected location to a second selected location over themap view (e.g., by overlaying route lines on the map view). Someembodiments automatically select a first one of the routes. The user canselect any of the other routes (e.g., by tapping over an unselectedroute), with the application remaining in state 1975 (but modifying thedisplay of the route lines to indicate the selection of the otherroute). In addition, when in state 1975, the application of someembodiments displays different UI controls related to routing andnavigation, including a direction list control, a navigation startcontrol, and others.

Also, various gestural interactions over the map on which the routes aredisplayed may cause the application to perform different modificationsto the map view (e.g., panning, rotating, zooming, modifying the mapperspective, etc.). When the integrated application receives gesturalinteraction over the map display while in the route display state 1975,the application transitions to state 1910 to perform gestural inputrecognition, with all of the gestural map modification operations (e.g.,corollaries to states 1915-1945) available. That is, the applicationtranslates the gestural input into panning, rotation, zoom, and/orperspective change operations similar to those described above forstates 1915-1945, with similar inertia and rebound features for thevirtual camera movement. Whereas the operations 1915-1945 return to themap browsing state 1905, the corollary operations accessed from theroute display state 1975 return to the route display state 1975.

In some embodiments, the route display state 1975 is accessible fromother states as well. For instance, if a user selects the quick-route UIcontrol on a banner while in state 1905, the application retrieves oneor more routes from the current location of the device to the locationwith which the banner is associated. In addition, some embodimentsdisplay previously requested routes among the search suggestions atstate 1960. When the user selects one of these suggested routes, theapplication transitions directly from state 1960 to state 1975 todisplay one or more routes over the map.

From the route display state 1975, the application can transition intovarious different modes depending on different controls selected by theuser. When the user selects a UI control to clear the routes, theapplication transitions back to state 1905 to display the map withoutany routes. In addition, the integrated application may enter one ormore navigation modalities from the route displaying state 1975.

When the selected route displayed at state 1975 starts at the currentlocation of the device and the user selects a navigation startingcontrol, the application transitions to the navigation state 1980. Insome embodiments, the application displays a cinematic transition fromthe map view into a more immersive 3D view for navigation. Within thenavigation state 1980 of some embodiments, a virtual camera follows thelocation of the user along the selected route in order to present theupcoming portions of the route. When either the route is completed (thedevice reaches the destination location) or the user selects a controlto end navigation, the application transitions to state 1905 to presentthe map browsing view 1905.

In some embodiments, various gestural interactions over the map on whichthe routes are displayed may cause the application to perform differentmodifications to the map view (e.g., panning, rotating, zooming,modifying the map perspective, etc.) while in the navigation mode 1980.In some embodiments, only some of the described map modificationoperations are available in the navigation mode. For instance, someembodiments allow the user to zoom in or out, but do not allow any othermodifications to the map. Thus, when the user provides gestural input,the gestural input recognition state 1910 filters out types of gesturalinput not associated with the zoom operation (and subsequently theapplication returns to state 1980). When the type of gestural inputassociated with the zoom operation is received, the gestural inputrecognition state recognizes this input and the application transitionsto a state similar to state 1925, for changing the zoom level of the map(with the inertia calculation, in some embodiments).

Other embodiments may enable different map modification operations. Forinstance, in some embodiments all of the gestural map modificationoperations (e.g., corollaries to states 1915-1945) are available whilein the navigation mode. Some embodiments allow a subset of the gesturalmap modification operations, such as zooming and a limited panningoperation. The panning operation of some embodiments, upon receiving thetype of gestural input associated with panning, moves the virtual camera(while in the navigation mode) to the side, then returns the virtualcamera back to pointing along the route. Whereas the operations1915-1945 return to the map browsing state 1905, the corollaryoperations accessed from the navigation state 1980 return to thenavigation state 1980.

When the selected route displayed at state 1975 starts at a locationother than the current location of the device (or the route is a walkingroute) and the user selects a navigation starting control, theapplication transitions to the stepping mode, or route inspection mode,at state 1985. In some embodiments, the application displays themaneuvers performed along the route one at a time (e.g., as navigationsigns). By providing gestural input (e.g., swipe gestures) to themaneuvers, the user can view the different maneuvers while in the routeinspection mode. The maneuvers are overlaid on a map and at least aportion of the route is displayed in the map.

As in the route display mode, various gestural interactions over the mapmay cause the application to perform different modifications to the mapview (e.g., panning, rotating, zooming, modifying the map perspective,etc.). When the integrated application receives gestural interactionover the map display while in the stepping mode 1985, the applicationtransitions to state 1910 to perform gestural input recognition, withall of the gestural map modification operations (e.g., corollaries tostates 1915-1945) available. That is, the application translates thegestural input into panning, rotation, zoom, and/or perspective changeoperations similar to those described above for states 1915-1945, withsimilar inertia and rebound features for the virtual camera movement.Whereas the operations 1915-1945 return to the map browsing state 1905,the corollary operations accessed from the stepping mode 1985 return tothe stepping mode 1985.

Furthermore, in some embodiments the gestural input recognitionrecognizes at least one type of gestural input over the displayedmaneuvers in order to switch between the maneuvers. When a particulartype of gestural input (e.g., a swipe gesture) is received over thedisplayed maneuver (as opposed to over the map view), the applicationtransitions to a state (not shown) for changing the displayed maneuver,then returns to state 1985.

When the integrated application receives gestural interaction over themap displayed while in the stepping state 1985, the applicationtransitions to state 1910 to perform gestural input recognition, withall of the gestural map modification operations (e.g., corollaries tostates 1915-1945) available. When the modification operations are done,the application returns to state 1985. When the user selects a controlto end stepping through the maneuvers, the application transitions tostate 1905 to present the map browsing view.

In addition, in some embodiments the application can transition from thestepping mode 1985 to an auto-stepping state 1990. When the user selectsa location tracking control while the application is in state 1985, theapplication transitions to an automatic stepping mode 1990, which is adifferent navigation modality. When in the automatic stepping mode ofsome embodiments, the integrated mapping, search, and navigationapplication displays the maneuver to which the device's location isclosest (e.g., as measured by a juncture at which the maneuver isperformed). When the device moves (e.g., along the route) to a locationcloser to a different maneuver, the auto-stepping mode automaticallydisplays the different maneuver. When the user deselects the locationtracking control, the application transitions back to the stepping mode1985. When the user selects a control to end navigation while in theauto-stepping state 1990, the application transitions to state 1905 topresent the map browsing view.

As in the stepping mode 1985, various gestural interactions over the mapmay cause the application to perform different modifications to the mapview (e.g., panning, rotating, zooming, modifying the map perspective,etc.). When the integrated application receives gestural interactionover the map display while in the auto-stepping mode 1990, theapplication transitions to state 1910 to perform gestural inputrecognition, with all of the gestural map modification operations (e.g.,corollaries to states 1915-1945) available. That is, the applicationtranslates the gestural input into panning, rotation, zoom, and/orperspective change operations similar to those described above forstates 1915-1945, with similar inertia and rebound features for thevirtual camera movement. Whereas the operations 1915-1945 return to themap browsing state 1905, the corollary operations accessed from theauto-stepping mode 1990 return to the auto-stepping mode 1990. Inaddition, some embodiments automatically turn the location trackingcontrol off when the user pans the map a particular distance, in whichcase the application returns to the stepping mode state 1985 rather thanauto-stepping state 1990.

II. Display of Navigation Signs

The above sections introduce the turn-by-turn navigation features ofsome embodiments. One such feature is the navigation signs provided bythe mapping application describing the different maneuvers for the userto perform. These signs may indicate turns, a distance over which tocontinue traveling straight, when to take a freeway off-ramp, or othermaneuvers for the user to perform. Some embodiments provide variousanimations for the signs, including showing the signs as passing overthe user location indicator in 3D mode, modifying the appearance of asign to indicate an upcoming maneuver, and using secondary signs whentwo maneuvers will be performed in rapid succession.

A. Realistic Look and Different Formats in Different Contexts

The navigation signs, in some embodiments, may have differentappearances in different contexts. Some of these differences aredescribed in greater detail further below. Specifically, graphicalindicators of maneuvers to perform (e.g., direction indicators that aredescribed further below) and instruction text describing those maneuversmay be adapted to fit the context of the navigation signs beingdisplayed. For example, different-sized signs may have either simple orcomplex maneuver descriptions, and instruction text may be adapted tothe size of the sign and may be based on other information displayedwithin the sign.

Some embodiments display the navigation signs in such a way as to givethe signs the appearance of a realistic road sign. Some embodimentsdisplay the navigation signs as rich, textured images (e.g., usingshadows, shading, etc.) as opposed to simply displaying a flat image onthe map display. In addition, some embodiments use shading for thenavigation sign that matches the color(s) of road signs in the areathrough which the application is navigating. The application also usesrealistic highway shields to mark roads in some embodiments. Forinstance, for numbered state and federal highways, the application willeither use the highway shield associated with the road within thenavigation sign (e.g., off to the side of the sign), replace the name ofthe road in navigation instructions with the highway shield, orotherwise include the highway shield in the graphical display.

FIG. 20 illustrates several GUI scenarios in which such highway shieldsare used. The first such scenario 2005 illustrates the mappingapplication in turn-by-turn navigation mode, showing an instruction toproceed straight on US-101 North for 20 miles. In this example, the roadsign for US-101 is displayed inline within the text instruction “Gostraight on US-101 North”, as a substitute for the actual text “US-101”.Some embodiments replace text names of roads with road signs when theroad has a sign and that sign is available as an image to the mappingapplication.

The second example 2010 illustrates the highway shield displayed on theright side of the navigation sign rather than inline in the textinstruction. This scenario illustrates an alternative display used bysome embodiments for the same instruction as in example 2005. Thehighway shield in this case is displayed as the same size as thegraphical indicator arrow on the left side of the navigation sign. Inaddition, because the information is presented in the road sign, theapplication removes the “on 101 North” portion of the text that wouldotherwise be present.

The third example 2015 illustrates the case in which the navigation signis shaded to match the type of road shown in the highway shield. In thisscenario, the instruction tells the user to go straight on CA-1 North.The “CA-1” is replaced with the highway shield sign for CA-1. While someembodiments shade this sign using green (the color of signs used forCalifornia state highways), other embodiments shade the navigation signusing the color of the road shield signs found along the actual highway.Other embodiments use green to match the color of road instruction signsfound above freeways in the region in which the device is located (e.g.,green for California).

The fourth scenario 2020 illustrates a merge maneuver onto Interstate-5within the navigation sign. Much like the first example 2005, thisillustrates the road shield sign as inline text. Furthermore, shading isused within the road shield in order to match the look of the actualinterstate signs, with the top portion shaded red and the bottom portionshaded blue. As mentioned, some embodiments instead shade the entirenavigation sign using a combination of these colors.

Although FIG. 20 does not illustrate different appearances of directionindicators 2090, the mapping application of some embodiments usesdifferent appearances in order to adapt the direction indicators to fitthe context of the navigation signs being displayed.

B. Dynamic and Animated Presentation of Signs

The above-described situations of FIG. 20 illustrate static display ofthe navigation signs (i.e., not showing any changes made to the signs).Some embodiments provide animation or other dynamic displays of thesenavigation signs. These displays include displaying the appearance ofthe sign passing over the head of the user representation (thenavigation puck) in the map display as a user makes a maneuver and thesign is removed. In addition, subtle animation may be applied to a signas a maneuver approaches in order to bring the upcoming maneuver to theuser's attention. Finally, when two maneuvers occur in short succession,the application displays the navigation sign for the second maneuver asqueued up behind the first sign.

1. Animated Removal and Presentation of Navigation Sign

FIG. 21 illustrates, over four stages 2105-2120, the animation of someembodiments for removing a navigation sign and introducing the nextsign. In some embodiments, the animation of the removed sign mimics thatof a road sign passing overhead on a highway. While this figureillustrates the animation within the context of 3D mode, someembodiments also include the animation in 2D mode. Other embodimentsspecifically provide the animation for 3D mode.

The first stage 2105 illustrates a navigation sign 2125 indicating amaneuver of merging onto Main St. for the user to perform in 100 ft. Thesecond stage 2110 illustrates the animation to remove the navigationsign 2125 as the user performs the maneuver. As the user physicallymerges onto Main St., the navigation sign 2125 enlarges and beginsdisappearing from the field of view, as would a road sign above afreeway. In some embodiments, the mapping application also applies aperspective tilt to the sign, to further mimic the appearance of thesign passing overhead.

At the third stage 2115, the subsequent navigation sign 2130 begins toappear from the horizon, or a closer approximation of the horizon. Someembodiments do not actually render the map all the way out to thehorizon in 3D mode, and start animating the upcoming navigation signfrom the distance at which the 3D rendering ends. This animation ismeant to resemble the approach towards a road sign on the freeway,though often at a faster speed (in order to quickly bring the navigationsign to full size, and avoid the distraction of a lengthy animation).The fourth stage 2120 illustrates the resultant display, with thesubsequent navigation sign 2130 displayed at the top of the screen inthe normal position.

In addition to the animations shown in FIG. 21, some embodiments alsoinclude more complex animations in some cases. As one example, someembodiments rotate a navigation sign as it leaves the display when auser makes a turning maneuver, in order to mimic the appearance of auser turning underneath the sign.

2. Occasional Emphasis

In some cases, the mapping application may display a navigation signwell before the maneuver described by the navigation sign will beperformed. For instance, if a user enters a freeway, and the nextmaneuver involves a freeway exit in 15 miles, the application maydisplay a navigation sign indicating the upcoming freeway exit wellbefore the user needs to begin preparing to actually exit the freeway.When it comes time to alert the user that the juncture at which toperform the maneuver is approaching, different embodiments use differenttechniques. Some embodiments include audio alerts, with the user deviceproviding voice navigation to indicate that the juncture is approaching.

Some embodiments, either in conjunction with the audio alert or wheneverthe audio alert is turned off, provide a visual indication that themaneuver is upcoming through the display of the sign. For instance, insome embodiments the application modifies the color of the sign (e.g.,from green to white or green to yellow) along with the color of thegraphical indicator arrow (e.g., from white to black). Other embodimentsdisplay a less obtrusive shimmer over the navigation sign, intended tocatch the user's attention without being overly obtrusive.

FIG. 22 illustrates such a shimmer animation over four stages 2205-2220.These stages illustrate the background of the display as gray, in orderto contrast with the shimmer as it moves across the sign (shown inwhite). The first stage 2205 illustrates a navigation sign 2225,currently indicating a right turn maneuver in 1000 ft.

At the second stage 2210, the right turn is now only 500 ft. away. Theapplication has judged that this is the appropriate distance at which toalert the user to the upcoming maneuver, and therefore has begundisplaying a shimmer across the navigation sign 2225. The third andfourth stages 2215 and 2220 illustrate the continuation of thisanimation. In some embodiments, the animation resembles a light beingmoved across the sign from left to right. Other embodiments display asimilar animation from right to left, or other such animations (e.g., alight radiating out from the center of the sign, etc.).

Some embodiments vary the distance from the maneuver at which theanimation begins based on various factors, such as the speed at whichthe device is moving (based on location tracking information) and thespeed limit of the road on which the user is currently traveling. Forexample, some embodiments have a set time before the intersection thatthe animation should be displayed, and use this speed information tocalculate the appropriate distance. Some embodiments also vary thedistance based on the type of maneuver being made (e.g., allowing moretime for exiting a freeway than for making a right turn off of aone-lane road).

3. Secondary Signs

When a route requires two distinct maneuvers in rapid succession, someembodiments display the navigation sign for the second maneuver asstacked underneath the navigation sign for the first maneuver. Thisalerts the user to the impending nature of the second maneuver. Whenseveral maneuvers will be performed in succession, some embodimentsstack more than two navigation signs on top of each other.

FIG. 23 illustrates the display of two signs for maneuvers in quicksuccession over four stages 2305-2320. In the first stage 2305, a firstnavigation sign 2325 indicates that the upcoming maneuver, at a distanceof 1000 ft., is a left turn onto East St. As this is a full sizeturn-by-turn navigation sign, the application displays a first type ofgraphical indicator arrow (i.e., a complex arrow) for this maneuver. Ascan be seen on the map with more careful review than may be available toa driver (who will mostly be looking at the road), a right turn ontoSouth St. will be required shortly after the left turn onto East St. inorder to follow the given route. In order to make this more apparent tothe user, the application displays a second navigation sign 2330underneath the first navigation sign 2325. The second sign includes asecond type of graphical indicator arrow (i.e., a simpler arrow) as lessspace is provided. Furthermore, less information is provided to the userin the second sign 2330.

The second stage 2310 illustrates that the user has now traveled 900feet, so that there are only 100 ft. from the left turn maneuver. Otherthan the updates to the distance in the navigation sign 2325 (and themovement of the 3D map), the display has not yet changed. The thirdstage 2315 illustrates the display immediately after the left turnmaneuver has been performed onto East St. As shown, the secondnavigation sign 2330 is now a full-sized navigation sign with a complexgraphical indicator arrow and additional textual information (a distanceof 50 feet and text instructions to turn right). Some embodimentsanimate the transition from the smaller sign to the full-size sign,while other embodiments simply replace one with the other.

The fourth stage 2320 illustrates the display after the user has madethe second maneuver (the right turn onto South St.). The application nowdisplays a navigation sign 2335 for the next maneuver, a left turn ontoWest St. Because this maneuver is 2.8 miles away, the application didnot stack sign 2335 under sign 2330. Because the navigation is in 3Dmode, some embodiments do display the animation described above byreference to FIG. 21.

In the above example, the application stacks signs for maneuvers thatoccur 50 feet apart, but does not stack signs for maneuvers that occurseveral maneuvers apart. The threshold distance for when to consider twomaneuvers subsequent may depend on a variety of factors. Someembodiments store a set distance that is not variable. Other embodimentslook at the type of roads involved in the maneuver (e.g., based on afunctional road class variable that describes the road in back-end mapdata) or the speed limits, assume a likely speed for the user after themaneuver, and set the threshold distance based on this data (i.e., basedon a threshold time between maneuvers, such as 30 seconds).

III. Dynamic Generation of Adaptive Directional Indicators

The above section described various features of the navigation signsprovided for turn-by-turn navigation. As shown, these navigation signsinclude directional indicators that graphically describe a maneuver forthe user to perform and, in some cases, also show a context for themaneuver by indicating other branches of the intersection at which themaneuver is performed. These directional indicators may appear invarious different contexts throughout the mapping and navigationapplication, both in various aspects of turn-by-turn navigation as wellas route inspection.

In order to generate the directional indicators, the mapping applicationreceives data about each of the intersections (also referred to in someplaces below as junctures) that indicates the different branches of theintersection and notes through which branches the maneuver enters andexits the intersection. In some embodiments, this intersectioninformation is stored by a mapping service that the mapping applicationaccesses in order to retrieve map data as well as route and navigationinformation. In this Section, subsection A initially describes thegeneration of this intersection data by the mapping service servers.Subsection B then describes the dynamic generation of the directionalindicators by the mapping application operating on the client device.

A. Server Side Generation of Intersection Data

The mapping service of some embodiments receives data specifying eachjunction of road segments. In some embodiments, the mapping serviceautomatically generates additional data for each of these junctionsstored in the map data, and in some embodiments converts the junctiondata into intersection data. This junction information simplifies eachreceived junction (or a set of related junctions that are viewed in thereal world as a single intersection) into a set of branches leaving thejunction at different angles. When a user requests a route through amapping application operating on a device, the device sends the requestto the mapping service, which generates a route from a starting locationto an ending location. The mapping service also generates turn-by-turnnavigation instructions for the route in some embodiments, using theintersection data. The mapping service identifies the intersections atwhich maneuvers are made, and modifies the intersection data to bespecific to the maneuver made during the route. This data is then sentto the user device on which the client mapping application runs. Thefollowing subsections first introduce the creation of navigation datafor a route, then discuss the precalculation of intersection data by themapping service (so that the data is available for use in generatingnavigation data), and then finally describe specific types ofmodifications made to the intersection data for a requested route.

1. Navigation Data Creation

FIG. 24 conceptually illustrates an operation performed by a mappingservice of some embodiments to generate a route for a requesting deviceand provide the route, with navigation instructions, to the requestingdevice. FIG. 24 illustrates this operation over six stages 2410-2460, inwhich the mapping service receives a request for the route, generatesthe route, generates intersection data for the route, and provides theroute to the device, which uses the information to display navigationinstructions.

As shown, each stage of FIG. 24 illustrates a device 2405 and a mappingservice 2400. The device 2405 may be a handheld device in someembodiments (e.g., a smart phone, tablet device, etc.), or may be adedicated navigation device (e.g., a navigation system built into anautomobile, a portable navigation device, etc.). In addition, in someembodiments, the device 2405 may be a non-portable device such as adesktop computer or other non-portable computing device.

The mapping service 2400 is a service to which the device 2405 connects(e.g., via a wired connection, wireless connection such as a cellnetwork, Wi-Fi, etc.) in order to request and receive map data, routedata, turn-by-turn navigation data, as well as additional information(e.g., information about places located on the map, etc.). As shown, themapping service 2400 stores map data 2415 and intersection data 2425,and includes a map generator 2435 and route generator 2445, among othermodules (not shown).

The map data 2415 provides data from which viewable map regions androutes can be generated. This map data, in some embodiments, includeslatitude and longitude data, name data, as well as descriptive dataabout roads and other pathways (e.g., walkways, ferry routes, bikepaths, etc.), natural features (e.g., rivers, lakes, mountain ranges,etc.), places of interest (e.g., buildings, businesses, parks, etc.),and other map items. In some embodiments, for example, a pathway isdefined as a series of latitude/longitude vertices, a name, anddescriptive data. This descriptive data may include a form of way (i.e.,whether the pathway is a single carriageway or a part of a dualcarriageway, whether the pathway is a one-way path), the class of roadto which the path belongs (e.g., motorway, local road, private road,bicycle path, etc.), as well as other information). In some embodiments,this map data is compiled by an outside source (i.e., a map provider)and provided to the mapping service, while in other embodiments themapping service provides its own map data. The map data may also be ahybrid of outsider-provided and internally-generated data. In addition,the map data may include geometry data for various map constructs, suchas roads, land cover, etc.

The intersection data 2425 provides pretabulated data for theintersections of road paths in the map data. In some embodiments, asdescribed below, the mapping service automatedly calculates intersectiondata for road pathway intersections using the map data. Thisintersection data 2425 may be stored by denoting an intersection type(e.g., point, roundabout) and a series of branches coming in and out ofthe intersection at different angles. While the map data 2415 and theintersection data 2425 are shown as separate storages, these may both bestored on the same physical storage or on separate physical storages,and the intersection data 2425 may in fact be part of the map data 2415.In addition, one or both of the map and intersection data might bedistributed across several physical storages (e.g., a series of disksfor storing the map data).

The map generator 2435 of some embodiments generates map information(e.g., map tiles) to transmit to the requestor device. The requestordevice requests a map for a particular region (e.g., usinglatitude/longitude information), and the map generator 2435 creates (oruses pre-generated) map tiles for the region, then sends data for thesetiles (e.g., as encoded vector and/or image data) to the device.

The route generator 2445 calculates optimal routes between two or morepoints in response to user requests. In some embodiments, the routegenerator 2445 calculates the routes based on the map data, usingoptimization algorithms. The routes may be defined as a series ofintersections, a series of road pathways, or in other manners. Inaddition, when a user requests a route, the route generator 2445provides intersection data for use by the device in turn-by-turnnavigation. In some embodiments, the intersection analyzer 2455retrieves intersection data 2425, and modifies this data for navigationof the route, as described below.

As shown, at stage 2410, the device 2405 sends a request for a route tothe mapping service 2400. In some embodiments, the user enters astarting address (or place) and an ending address (or place),potentially including additional midpoint locations (e.g., starting atA, going to B, then going to C from B). The device then transmitslocation information to the mapping service. In some embodiments, thedevice translates the locations into latitude and longitude data, whilein other embodiments this conversion is performed by the mappingservice.

At stage 2420, the route generator 2445 accesses the map data 2415 inorder to generate one or more routes for the series of locations. Insome embodiments, the route generator 2445 uses an optimizationalgorithm to find the best (and second best, third best, etc.) routethat connects the series of locations.

At stage 2430, the intersection analyzer 2455 identifies maneuvers alongthe route for which navigation directions need to be generated andretrieves intersection information for these maneuvers. Some embodimentsgenerate turn-by-turn navigation directions to provide to the devicealong with the route. To generate these directions, the mapping service2400 identifies each time the route changes pathways, at which point theuser following the directions will have to perform a maneuver (e.g.,right turn, slight left turn, U-turn, merge, etc.). In some embodiments,each of these pathway changes corresponds to a pretabulated intersectionstored in the intersection data 2425. The intersection analyzer 2455retrieves this intersection data. In some embodiments, each intersectionis stored as a series of branches coming out of the intersection atvarious angles (e.g., based off of North=0°). In some embodiments, inaddition to the intersection data, the route generator creates routingdirections, that generally describe the maneuver to be performed.Examples of such descriptions include “turn left”, “highway off ramp”,“U-turn”, etc. In other embodiments, this description is derived by theclient mapping application based on the received intersection data.

Next, at stage 2440, the intersection analyzer 2455 generatesintersection information designed for the route. In some embodiments,this entails modifying the angles to set the direction of travel intothe junction to 0° (i.e., setting the branch on which the route entersthe junction to 180°). This effectively rotates the intersectiondescription by the difference between due North and the route's incomingdirection of travel. In addition, the intersection analyzer 2455 tagsone of the branches as the exit branch. Some embodiments tag an entrancebranch as well, while other embodiments rely on the device to identifythe 180° branch as the entrance branch.

Stage 2450 illustrates that the mapping service 2400 then transmits(e.g., via the same network that the device used to transmit the routerequest) the route data (i.e., route data and intersection data fornavigation) to the device 2405. As shown at stage 2460, the device 2405then uses the intersection and route data generated by the mappingservice to display navigation instructions for the user of the device.In some embodiments, the navigation instructions include a display ofthe intersection along with a stylized arrow showing the maneuver (inthis case, a right turn) through the intersection.

While the mapping service 2400 is displayed as including a map generatormodule and a route generator module, one of ordinary skill in the artwill recognize that the mapping service may include additional modules,or different breakdowns of modules. The mapping service may consist of asingle computing device (e.g., a server) storing all of thefunctionality and data, or the functionality may be distributed betweenmultiple servers (e.g., one process on a first server and a secondprocess on a second server, numerous servers that perform the sameoperation in parallel for different users, or other configurations ofcomputing devices that perform the functionality described herein).

FIG. 25 conceptually illustrates a process 2500 performed by the mappingservice of some embodiments in order to generate and transmit route andintersection data to a user. As shown, the process 2500 begins byreceiving (at 2505) a request for a route between two locations on amap. In some embodiments, when the user requests a series of more thantwo locations, each segment is treated as a separate route (i.e., frompoint A to point B is a first route, then point B to point C is a secondroute).

The process then generates (at 2510) at least one route between thelocations using map data. In some embodiments, the process uses anoptimization algorithm to identify the best (or two best, three best,etc.) routes between the two locations. These routes may be described asa series of vertices along pathways, a series of intersections betweenpathways, or with another description.

With the routes generated for the start and end locations, process 2500selects (at 2515) one of the generated routes in order to createturn-by-turn instructions for the route. The process then identifies (at2520) maneuvers to make along the route. In some embodiments, themapping service identifies each time the route changes pathways, atwhich point the user following the directions will have to perform amaneuver (e.g., right turn, slight left turn, U-turn, merge, etc.).

Next, the process retrieves (at 2525) intersections for each of themaneuvers. In some embodiments, each of these pathway changescorresponds to a pretabulated intersection stored by the mappingservice. The generation of these intersections is described in detailbelow. In some embodiments, each intersection is stored as a series ofbranches coming out of the intersection at various angles (e.g., basedoff of North=0°). In addition, the intersection data stores the type ofintersection in some embodiments (e.g., point, roundabout, trafficsquare, etc.).

The process then modifies (at 2530) the intersection information foreach of the maneuvers. In some embodiments, this entails modifying theangles to set the direction of travel into the junction to 0° (i.e.,setting the branch on which the route enters the junction to 180°). Thiseffectively rotates the intersection description by the differencebetween due North and the route's incoming direction of travel. Inaddition, some embodiments tag one of the branches as the exit branch.Some embodiments tag an entrance branch as well, while other embodimentsrely on the device to identify the 180° branch as the entrance branch.

The process 2500 next determines (at 2535) whether additional routesremain for which to generate maneuver/intersection information. Whenadditional routes remain, the process returns to 2515 to select the nextroute. Different variations of routes from a start location to an endlocation may overlap in part, in which case some embodiments reuse thealready-generated set of intersections for the overlapping portions.

Once intersections are generated for all of the routes, the processtransmits (at 2540) the route and intersection information to therequestor (e.g., a requesting device). As mentioned, the requestingdevice uses this information in some embodiments in order to generateturn-by-turn navigation, including stylized junction/maneuver arrows.

2. Precalculation of Intersection Data

As mentioned above, some embodiments precalculate intersection data fromthe stored map data (e.g., road segment and junction data). The mappingservice then stores this intersection data for use in generatingturn-by-turn navigation instructions. The following section describesseveral processes used to generate this intersection data, in which themapping service receives vendor-provided junctions, identifies whetherany sets of the received junctions should be consolidated into a singleintersection, identifies pairs of road segments that should be joinedtogether within an intersection, and generates angles for theintersection. Within this section, the term junction will be generallyused to refer to vendor-provided information at which two path segmentsintersect, while the term intersection will refer to data generated fromthe junctions that represents where two or more roads meet in the realworld. Thus, multiple junctions might be consolidated into oneintersection, and junctions between two road segments that are actuallyjust a continuation of a single road might not be consideredintersections at all, in some embodiments.

The following represents pseudocode of some embodiments for generatingintersection data for point intersections:

-   -   Identify all internal segments;    -   Identify all internal turn channels and mark them as internal        segments;    -   For each internal segment:        -   Gather all contiguous internal segments;        -   Mark the gathered internal segments as processed;        -   Build an intersection from this collection of internal            segments;

In addition to other data (e.g., locations of parks, waterways,businesses, etc.), the map data stores information about pathways (i.e.,roads, walkways, bike paths, etc.). Each pathway, in some embodiments,is received from a map provider as a series of segments (e.g., roadsegments). For a given segment, in some embodiments the stored dataincludes start and end junctions for the segment, geometry data thatdefines the course taken by the path between the start and endjunctions, a path characterization (or “form of way”), a direction oftravel (which may, in some embodiments, involve a one-way flag), one ormore names for the path (e.g., “I-405 S”, “San Diego Freeway”, etc.), aclass that indicates the level of importance of the path, and a netclass(a connected graph of paths to which the path belongs). In someembodiments, the geometry information comprises a series oflatitude/longitude vertices through which the path travels. The form ofway attribute, in some embodiments, includes the followingcharacterizations: single carriageway, dual carriageway, motorway, sliproad, connector, walkway, stairs. Some embodiments may includeadditional characterizations.

FIG. 26 conceptually illustrates a process 2600 of some embodiments fordetermining path segments between sets of junctions that should betreated together as a single intersection. As shown, the process 2600begins by receiving (at 2605) a junction between at least two pathsegments (e.g., road segments). In some embodiments, the mapping servicereceives (e.g., as precalculated data from a map vendor) a set of roadsegments and a set of junctions. Each road segment follows a pathbetween two such junctions, and each junction references at least tworoad segments that enter and/or exit the junction. On the other hand, insome embodiments, the junctions are not received from the map datavendors and the mapping service traverses the path data to identifyintersections between paths and analyzes these intersections in order topretabulate the junctions.

The process then determines (at 2610) whether any of the path segmentsat the received junction are dual carriageways. As mentioned, a dualcarriageway is a path characterization used in some forms of map data.Many roads that are divided (e.g., by a median, a double-yellow line,etc.) are received (and drawn) as two separate path segments, one foreach direction. Each of the path segments is then marked with adirection of travel and as one-half of a dual carriageway. Because auser will typically think of an intersection of two roads that are bothdivided by medians as a single intersection (rather than four separateintersections), the junction generation process attempts to unify thesefour received junctions into a single intersection to present to a userfor navigation purposes.

When none of the path segments are marked as dual carriageways, theprocess calculates (at 2615) the intersection branches using only thepath segments specified in the received junction (i.e., the intersectionwill include only the one received junction). In some embodiments, thecalculation of junction branches entails calculating the angle at whicheach of the segments specified for the junction leaves the junctionlocation. The process then ends. FIG. 27 illustrates an example of sucha junction 2700, also illustrating that there is no requirement that thepath segments meet at right angles or that the paths continue in astraight line through the junction.

When at least one path segment specified for the received junction is adual carriageway, the process determines (at 2620) whether there existsa cross-traffic turn off of a dual carriageway at the junction. Across-traffic turn is a turn off of the dual carriageway in a directionthat will cross through the matching half of the dual carriageway (i.e.,the other direction of the road), assuming it exists. In the UnitedStates, a left turn is a cross-traffic turn. While the examples shown inthis document involve right-handed driving (i.e., driving on the rightside of the road), one of ordinary skill will recognize that theexamples are equally applicable to left-handed driving areas (e.g.,England) as well. FIG. 28 illustrates an intersection 2800 that includestwo dual carriageway paths 2805 and 2806 and a one-way road 2810. At thejunction 2815, there is no cross-traffic turn off of a dual carriageway,because the only options are a right turn off of the the dualcarriageway path 2805 or a left turn off of the one-way street 2810.When no such turn exists, the process 2600 stores (at 2625) the receivedjunction while recognizing that it may still be part of a largerintersection, in order to determine whether to include the receivedjunction with other received junctions (e.g., the junction 2820 betweenthe one-way road 2810 and the dual carriageway path 2806) in a largerintersection. For instance, in the intersection 2800, the process willwant to join the received junction 2820 with the received junction 2815into a single larger intersection. The process then ends.

When a cross-traffic turn off of a dual carriageway exists at thejunction (for instance, at junction 2820), the process moves (at 2630)in the direction of the cross-traffic turn until the next dualcarriageway path is reached. In some embodiments, because the pathsegments start and stop at junctions, the next dual carriageway pathwill be reached at a different received junction (though not necessarilythe next junction, if a road such as a left turn lane is received as aseparate path segment). For instance, from intersection 2820, theprocess would traverse the path 2810 away from the junction 2820 untilreaching the next dual carriageway, at junction 2815.

The process 2600 then determines (at 2635) whether the dual carriagewaypath reached at 2630 has a direction of travel in the opposite directionof the originating dual carriageway path. This, essentially, is a quickdeterminant of whether the second dual carriageway could be the matchingpath for the first dual carriageway (i.e., whether they are likely to betwo sides of the same road). In most cases, this next dual carriagewaywill be the matching path, due to the nature of how roads are typicallybuilt.

In the case when the second dual carriageway is not in the oppositedirection of the originating path, the process proceeds to 2625 to storethe newly reached junction for later use in determining whether toinclude it with any other received junctions. For example, if the leftturn off of path 2806 reached another dual carriageway with a downwarddirection of travel, then path 2806 could be assumed to not have a matchin the data (as far as the junctions are concerned, at least), but thenewly identified path might itself have a match.

On the other hand, if the two dual carriageways have opposite directionsof travel, the process identifies and stores (at 2640) the segmenttraversed by the cross-traffic turn. In the example of FIG. 28, thesegment from junction 2820 to junction 2815 would be stored. Thissegment will be used as part of additional junction consolidationprocesses in some embodiments. The process then ends.

The above process 2600, when applied to all the junctions within a mapregion, will generate a set of segments between dual carriageways. Someembodiments use these segments to link together received junctions andidentify additional received junctions to include in a singleintersection definition. The following represents pseudocode of someembodiments for identifying all such “internal” segments for a complexintersection:

For each segment that is a dual carriageway;

-   -   For each connection with cross-traffic turn where a path can be        assembled to other side of intersection;        -   Mark all segments on the path to other side as internal            segments;

This pseudocode includes a determination as to whether a path can beassembled to the other side of an intersection from a segment. Thefollowing includes pseudocode of some embodiments for such adetermination:

-   -   Add first segment to path;    -   Get connections from last segment on path;    -   Iterate through each connection to either find a connection to        other side or find connection that is best continuation;        -   If connection is other side, note success and end;        -   If no connection is other side and no connection is the best            continuation, note failure and end;        -   Otherwise:            -   Add segment to end of path;            -   If path is too far, note failure and end;            -   If too many crossings, note failure and end;            -   Otherwise return to get connections for added segment                and iterate through connections;

FIG. 29 conceptually illustrates a process 2900 for linking togetherseveral junctions into a single intersection and identifying thebranches of the intersection. The process 2900 begins by receiving (at2905) a set of intersecting segments between dual carriageways. Thesesegments may be identified using a process such as that shown in FIG.26, in some embodiments. The mapping service then groups together setsof such segments that intersect each other (i.e., at receivedjunctions). FIG. 30 illustrates a commonly existing intersection 3000,between a dual carriageway with paths 3005 and 3006 and a dualcarriageway with paths 3010 and 3011. The set of intersecting segmentsare shown in this figure as thicker lines.

The process then identifies (at 2910) all junctions and path segmentsthat connect directly to the set of intersecting segments at junctions.That is, the set of intersecting paths intersect at junctions, but thesejunctions may contain additional path segments. For instance, in theexample intersection 3000, the eight dual carriageway path segments thatleave the intersection all intersect with the internal (thicker) pathsegments at the four junctions. Thus, the four junctions and eightexternal path segments are all included in the intersection.

FIG. 31, on the other hand, illustrates an intersection 3100 in whichleft-turn channels are defined as separate path segments. In this case,because the left-turn channels intersect the internal segments atjunctions received in the initial map data, these channels areidentified by the process 2900. The left-turn channels may becharacterized in the map data as slip roads or single carriageways, inmost cases.

The following represents pseudocode of some embodiments for identifyingall turn channels to treat as “internal” to an intersection:

-   -   For each segment that is a dual carriageway;        -   For each connection with cross-traffic turn where a path can            be assembled to internal segments;            -   Mark all segments on the path to the internal segments                as internal segments themselves;

This pseudocode includes a determination as to whether a path can beassembled to the internal segments from a segment (e.g., a turnchannel). The following includes pseudocode of some embodiments for sucha determination:

-   -   Add first segment to path;    -   Get connections from last segment on path (i.e., segments        connected to last segment at junction);    -   Iterate through each connection to either find either an        internal segment or find connection that is best continuation;        -   If connection is an internal segment, note success and end;        -   If no connection is internal segment and no connection is            the best continuation, note failure and end;        -   Otherwise:            -   Add segment to end of path;            -   If path is too far, note failure and end;            -   If too many crossings, note failure and end;            -   Otherwise return to get connections for added segment                and iterate through connections;

Next, the process 2900 defines (at 2915) a new intersection as includingall of the identified junctions and path segments, including those thatdirectly intersect the initial set of path segments. In someembodiments, in the case illustrated in FIG. 31, the junctions where theleft-turn channels leave their originating dual carriageway segmentswould be included as well as the left-turn channels that intersect theinitial segments. In this situation, some embodiments identify the otherjunction (i.e., the start junction) for the slip road or singlecarriageway path segment, which will be where the path segmentintersects with one of the dual carriageway path segments beforeentering the intersection. When the single carriageway path segmentstays internal to a (presumed) pair of dual carriageway paths for athreshold distance (e.g., 1 km), some embodiments assume the pathsegment is a part of the road defined by the dual carriageway paths, andeliminate the junction from consideration.

When processing a slip road or other connector outside of the dualcarriageways (e.g., the slip road 3205 shown in the intersection 3200 ofFIG. 32), some embodiments do not treat the slip road as a path into thedual carriageway intersection. Instead, some embodiments identify thepath characterization as a slip road and attempt to form a closed loopincluding the start and end junctions of the slip road. When this closedloop shares a common junction with the newly defined intersection (aswill typically be the case), the slip road may be associated with theintersection but not treated as an internal path of this intersection.On the other hand, when the newly defined dual carriageway intersectionhas grown due to the presence of, e.g., left-turn channels, such thatthe slip road junctions are encompassed by the intersection nowincluding the intersecting single carriageways (as for the slip road3305 in the intersection 3300 of FIG. 33), some embodiments treat theslip road as internal to the newly defined intersection. In thedescription of the intersection, these left turn channels, slip roads,etc., will typically be eliminated, as a user generally will not wantcomplex instructions, but will instead want an instruction of “make aright turn onto San Vicente Blvd” or something similar.

With the set of segments and junctions that form the intersectiondefined, the process needs to merge dual carriageways into singlejunction branches. The process 2900 next defines (at 2920) the set ofall paths entering the intersection, and defines (at 2925) the set ofall paths exiting the intersection. For a dual carriageway, which is aone-way path (half of a two-way road), the path will typically have anexit side and an entrance side. For purposes of merging, someembodiments treat each segment (the segment exiting the intersection andthe segment entering the intersection) as separate paths. Singlecarriageways that are not internal to dual carriageways (e.g., theadditional two-way path 3405 in the intersection 3400 of FIG. 34) willtypically be treated as separate branches and are not part of themerging analysis in some embodiments.

Next, the process determines (at 2930) whether the set of entrance pathsincludes any unpaired dual carriageway paths. When no such paths remainin the set (or none existed in the first place), the process stores (at2935) any unpaired dual carriageway left in the set of exit paths asseparate branches of the junction. In general, this will happen in thecase of mislabeled map data (the road is actually a one-way street) ormerging criteria that are too strict (leaving a pair of entrance andexit paths unmerged).

When an unpaired entrance path exists, the process selects (at 2940) oneof the entrance paths. The process then determines (at 2945) whether apotential match exists in the exit set. A potential match, in someembodiments, is a dual carriageway found by traversing the previouslyidentified segment to the left (to the right, in the case of left-handeddriving regions), or traversing the intersection in a clockwise fashion.

When no potential match exists (e.g., the next identified dualcarriageway in the traversal is also an entrance path, or the exit setis empty), the process stores (at 2950) the entrance path as a separatebranch of the intersection and then returns to 2930 to find the nextunpaired entrance path. On the other hand, when a potential matchexists, some embodiments determine (at 2955) whether the potential pairsatisfies a set of dual carriageway match criteria. These are criteria,in some embodiments, to determine whether a pair of dual carriagewaysare actually the two sides of the same road. Some embodiments determinewhether the two paths (1) are within a threshold distance (e.g., 25 m,50 m, etc.) where the paths enter/exit the intersection, and (2) whetherthe angles at which the paths hit their junctions within theintersection is within a threshold range of each other (e.g., 5°, 10°,etc.). To calculate the angle, some embodiments use the vertex closestto the edge of the intersection (or the location of the junction atwhich the path segment intersects the other segments within theintersection) and a vertex located a particular predefined distance(e.g., 50 m) away. The process then calculates the angle off of Northfor the line between the two vertices.

In some embodiments, the mapping service additionally looks at the namesof the paths to determine whether these match. When the names match,such embodiments may relax the geometry criteria for a matching pair(i.e., allow a greater distance between the paths or a greaterdifference in angles between the paths). Matching names might be, e.g.,“CA-1 South” and “CA-1 North”, or if both paths include “Wilshire Blvd.”as one of their names. Some embodiments may also look at the road classdata for confidence in matching dual carriageways.

If the two paths match, the process merges (at 2960) the paths into asingle branch of the newly defined intersection. As indicated above,intersections are stored as a set of branches at different angles. For amerged path, some embodiments store the angle as the average of theangles of the two paths that make up the branch. FIG. 35 illustrates thereduction of an eight-path intersection 3500 into four branches, inwhich the angle of the right branch 3510 is at half the offset fromhorizontal as the right exit path 3505, because the right entrance pathis on the horizontal. As shown conceptually, directions (entrance/exit)are not stored for intersection branches in some embodiments. Themapping service generates the routes using map data, which includes theintersections as well as directions of travel for the roads, so a routewill not travel the wrong way on a branch of the intersection.

On the other hand, when the paths do not match, the process stores (at2965) each of the paths as separate branches of the intersection. FIG.36 illustrates the reduction of a different eight-path intersection 3600into five branches. In this case, the dual carriageway paths 3605 and3606 on the right side do not merge and are therefore treated asseparate branches 3610 and 3611 of the reduced intersection. In thisexample, the angle at which each of these branches leaves theintersection is the angle that is stored for the branch (with noaveraging). The process 2900 then returns to 2930 to determine whetherany entrance paths remain. As stated, once the entrance path set isempty, the process proceeds to 2935, and subsequently ends.

The following represents pseudocode of some embodiments for generatingthe data for an intersection once the internal segments have beenidentified for the intersection (e.g., operations performed by some orall of process 2900):

-   -   Gather all external segments that touch internal segments for an        intersection;    -   Identify external segments that are surrounded by internal        segments in the intersection and mark them as internal;    -   Group together pairs of incoming and outgoing segments that        represent same road;    -   Compute an outgoing angle for each pair and for each unpaired        road;    -   Construct a template Intersection Pattern with one branch for        each angle;    -   If pattern exists for previously generated intersection, use        existing pattern to save space (refer intersection to existing        pattern);    -   Else if pattern does not exist, create and store new entry for        pattern;

As indicated, some embodiments store each intersection as a datastructure. This data structure indicates the branches of theintersection and the angles at which the branches enter and/or exit thejunction. FIG. 37 conceptually illustrates an example of such a datastructure 3700 of some embodiments for a point type intersection. Asshown, the intersection includes an intersection ID (which, in someembodiments is a unique identifier), a map data association, and a setof branches with angles and types. The map data association, in someembodiments, associates the intersection data structure with an actuallocation within the map. In some embodiments, this is simply alatitude/longitude point, but may also consist of other data in otherembodiments (e.g., a list of the path segments or aggregate paths thatmeet at the intersection). Each branch includes a type and an angle. Thetype, in some embodiments, is an intersection type. Some embodimentsdefine two intersection types: point and roundabout. However, otherembodiments may include additional intersection types, such as trafficsquares. Some embodiments store the intersection type as a property ofthe intersection rather than separately for each branch, but otherembodiments recognize the possibility of an intersection partially beinga roundabout but partially being a point intersection. The datastructure 3700 includes four branches, at the cardinal directions of 0°(North), 90° (East), 180° (South), and −90° (West). In some embodiments,the intersection data structure also includes references to anyjunctions (i.e., data received from the map data provider) and pathsegments that are contained within the defined intersection. For atypical intersection of two dual carriageways, four junctions arereferred to by such a data structure.

FIG. 38 illustrates a data structure 3800 of some embodiments for aroundabout intersection. Some embodiments provide specialized processingfor roundabout intersection. The following represents pseudocode of someembodiments for generating intersection data for roundaboutintersections:

-   -   Identify all roundabout segments;    -   For each roundabout segment:        -   Gather all contiguous roundabout segments;        -   Mark the gathered roundabout segments as processed;        -   Build a roundabout intersection from this collection of            roundabout segments;

In some cases, the map data identifies a roundabout (e.g., as a form ofway or through another indicator). This allows the mapping serviceintersection calculator to begin its specialized automated roundaboutprocessing. Specifically, when performing roundabout processing, themapping service attempts to identify pairs of flare connectors (i.e.,the portions of a road that flare into and out of a roundabout). In someembodiments, the intersection calculator traverses the roundabout (e.g.,in a counterclockwise fashion for right-handed driving) looking for anexit path that is followed, within a particular distance (e.g., angulardistance), by an entrance path. The process then determines whether tocombine these paths, looking at factors similar to those for mergingdual carriageways at point intersections. For instance, the factors usedmight include whether the names are similar, whether the distancebetween the exit/entrance paths is small enough, and potentially otherfactors. As a result of this processing, when a random road intersectsthe roundabout in between what otherwise appears to be an entrance/exitcombination, some embodiments treat this as three separate branches.

In order to calculate the angles of the branches, some embodimentsdetermine where the branch intersects the roundabout, rather than theangle of approach of the road. For entrance/exit combinations, theprocess takes the average of the two paths. FIG. 39 conceptuallyillustrates the reduction of a roundabout intersection 3900 tointersection data. The top path, despite approaching at approximately a30° angle off of North, is designated as a 0° branch—the user isprimarily interested in the distance around the traffic circle for theintersections, rather than the angle at which they enter and exit. Theother three branches are also designated cardinal directions, becausetheir flares average out to these directions. The data structure 3800shows the data structure for the roundabout junction 3900. Otherembodiments, however, use the angle at which the paths enter or exit theroundabout, rather than the distance around the roundabout at which thepaths intersect it.

The following represents pseudocode of some embodiments for generatingthe data for a roundabout intersection once the roundabout segments havebeen identified for the intersection:

-   -   For set of roundabout segments that form a simple loop:        -   Gather all non-roundabout segments that touch the loop,            ordered by the direction of travel around the loop;        -   Group together pairs of consecutive roundabout exit/entry            segments that represent same road;        -   Assign an angle to each pair and each unpaired segment;        -   Subtract the smallest angle from all angles (so smallest            angle=0);        -   Construct a template intersection pattern with one branch            for each angle;        -   If pattern exists for previously generated intersection, use            existing pattern to save space (refer intersection to            existing pattern);        -   Else if pattern does not exist, create and store new entry            for pattern;

As indicated in the above examples of pseudocode, some embodimentsperform additional compression when storing the intersections. The realworld contains millions (or hundreds of millions) of individualintersections, but many of these intersections have the sameconfiguration (especially when very small angular variations aretolerated). Thus, rather than storing separate data for each of thehundreds of millions of intersections, some embodiments utilizecompression in storing the intersections. As each intersection isprocessed, some embodiments store a template pattern for thatintersection. When additional intersections with the template patternare identified, such embodiments store a reference to that pattern(while still creating a separate data structure, as the locationinformation is different for two intersections that follow the samepattern).

3. Modification of Junction Data for Navigation

The above section described the generation of complex intersection data,typically done as an offline process prior to route generation. However,at the time of route generation, some embodiments modify theintersection data for transmission to the user. The mapping serviceproviding the route data modifies the angles to make them relative tothe direction of entry and marks one of the branches as an exit branch.

FIG. 40 conceptually illustrates a process 4000 of some embodiments formodifying intersection data in order to provide navigation informationfor a route. As shown, the process begins by receiving (at 4005) a routefor which to generate intersection information. As mentioned above, someembodiments generate one or more routes for each set of starting andending locations requested by a user device. Each of these routesconsists of a series of maneuvers at various path intersections (i.e.,at road intersections).

As shown, with the route identified, the process 4000 selects (at 4010)the next intersection along the route. Some embodiments begin with thefirst intersection (i.e., the first maneuver a user following the routewill make), starting from the start point of the route. Many routesinvolve long stretches along a particular road, going straight throughnumerous intersections (possibly including junctions of two roadsegments that are part of the same road and at which no other roadsintersect). In some embodiments, the navigation instructions do notinclude information about the intersections at which no turning maneuveris made. Accordingly, the next intersection is actually the nextintersection along the route at which a maneuver will be made.

The process then retrieves (at 4015) precalculated intersection data asa set of branches with associated angles. As described above, someembodiments store a data structure for each intersection, which liststhe branches of the intersection along with angles for each branch.FIGS. 37 and 38 illustrate examples of such data structures, for both apoint intersection and a roundabout intersection.

After retrieving the data structure for the selected intersection, themapping service rotates the intersection definition to normalize thedefinition to the direction at which the route enters the intersection.Accordingly, the process 4000 identifies (at 4020) the entry branch ofthe intersection and sets the entry branch to a predetermined angle.Some embodiments set the direction of movement into the intersection as0°, and therefore set the entry branch of the intersection to 180°.

The process then rotates the other branches of the intersection. Asshown, the process selects (at 4025) a next branch of the intersection.In some embodiments, the branches and angles are stored in an array,list, or similar data structure, and the process traverses this datastructure. The process sets (at 4030) the angle of the selected branchbased on an angular distance from the entry branch. For example, if theentry branch was stored as 0° (i.e., pointing North), then a branchstored as 95° will be shifted 180° to −85°. In addition, the processdetermines (at 4035) whether the selected branch is the exit branch ofthe junction (i.e., the branch at which the route exits theintersection). In order for the turn-by-turn navigation instructions atthe client mapping/navigation application to properly display themaneuvers, the device needs to know along which branch of theintersection the route exits. Thus, when the selected branch is the exitbranch, the process 4000 marks (at 4040) the selected branch as such.The process then determines (at 4045) whether any additional branches ofthe intersection remain to be converted for the route. When additionalbranches remain, the process returns to 4025 to select the next branchof the junction. When all branches have been processed for the currentintersection, the process 4000 determines (at 4060) whether additionalintersections remain along the route that need to be modified. Whenadditional intersections remain, the process returns to 4010 to selectthe next intersection. When the last intersection is modified, theprocess ends.

FIG. 41 illustrates a conceptual drawing of a route taken through anintersection 4100, a data structure 4105 for the intersection, and themodification of the data structure to create a new data structure 4110for turn-by-turn navigation instructions. As shown, the route entersfrom the right side (the 90° branch) and exits the intersection at thebottom (the −162° branch). In the modified data structure, the entrybranch has been rotated to 180°, causing a 90° rotation of the otherbranches. The branch at 18° rotates to 108°, the branch at −65° rotatesto 25°, and the branch at −162° rotates to −72°. In addition to therotation angles, the data structure 4110 has the last branch marked asthe exit for the navigation. Some embodiments include a binary exitfield, with the exit branch marked with a ‘1’ and all other branchesmarked with a ‘0’.

B. Client Side Dynamic Generation of Adaptive Displayed GraphicalIndicators

The above section describes the generation of the juncture (i.e.,intersection) data for use in turn-by-turn navigation. However, once theuser device receives this juncture data, the mapping client applicationoperating on the device must dynamically generate graphical maneuverindicators based on the juncture data in order to provide intuitiveturn-by-turn navigation for a route.

1. Example of Different Indicators in Different Contexts

In a navigation system, when presenting a user with graphicalrepresentations of upcoming maneuvers, there are two competing goals tosatisfy, namely the completeness of the representation and the clarityand legibility of the representation. The mapping application of someembodiments uses a novel adaptive mechanism to balance these goalsdifferently according to context.

For a currently displayed instruction, in the context of full-screenturn-by-turn navigation, the mapping application of some embodimentsabstracts a maneuver down to two elements: a prominent stylized arrowroughly representing the path of the vehicle through the juncture, and ade-emphasized set of lines and curves corresponding to other elements ofthe juncture. For instance, a right turn at a T-junction is representedby a large arrow with a right-angle joined with a smaller, dimmersegment that runs parallel to one of the large arrow's segments. Thesmaller segmented will also be pushed off to the side so that the pathtaken by the vehicle through the juncture dominates the display. Such arepresentation of a maneuver which includes an arrow with junctioncontext provides fairly complete information about the maneuver whileremaining abstract and easily understandable.

An alternate representation of a maneuver may omit the juncture contextentirely and simplify the primary arrow indicating the maneuver. When auser looks at maneuvers beyond the current maneuver (the next maneuverto make), the more detailed graphical representation may provide moreinformation than is required and be harder to read with a quick glance.For example, even if there is space to display the junction context fora secondary instruction that follows the current maneuver, someembodiments display only the simplified arrow for clarity. This adaptiveapproach also benefits space-constrained UI elements. While multitaskingor looking at lists of instructions, for example, the mappingapplication of some embodiments draws the simpler maneuver abstractionin order to produce something more easily discernible in a smaller area.

FIG. 42 illustrates several different scenarios in which the mappingapplication displays different types of graphical indicator arrows tovisually represent maneuvers to a user. The first scenario 4205illustrates route directions shown in a list view. The list viewdisplays a series of turn-by-turn instructions to get from a startlocation to an end location. In some embodiments, the user can view theturn-by-turn instructions without actually entering a navigation mode,or even following the route. In this situation, some embodiments displaya simple version of the graphical indicators for each turn. This is donefor space-saving reasons, as well as the fact that when the user is notactually approaching a maneuver, the intersection context is notespecially helpful.

The second scenario 4210 illustrates turn-by-turn navigation when theuser device on which the mapping application operates is locked. Asdescribed in detail below, the application is able to displayturn-by-turn navigation instructions even when the device is locked, inorder to continue providing instructions to the user. In this scenario,as shown, a simplified arrow is also displayed in some embodiments. Thisprovides a simple graphical indication of the turn within the lockscreen (in this case, a right turn), without providing the context datathat might be difficult for a user to pick out in the lock screen.

The third scenario 4215 also illustrates turn-by-turn navigation whenthe mapping application is not open (or not presently displayed) on thedevice on which the application operates. As described in detail above,the application displays turn-by-turn navigation instructions within thenotification banner space when the mapping application is not displayed.Much like in the lock-screen mode, the mapping application uses a simplegraphical indicator for the indicated maneuver (in this case a leftturn). Due to space constraints and the reasons described above for thelock-screen mode, the simple graphical indicator is used.

The previous three scenarios illustrate situations in which the simplegraphical indicators are used. One of ordinary skill in the art willrecognize that in some embodiments, the more complex stylized junctureplus maneuver graphical indicators might be used in the abovesituations. The following three scenarios illustrate indications inwhich these more complex indicators are used.

The fourth scenario 4220 illustrates route overview directions, in whichthe user can view an entire route from a starting location to endinglocation. The user can swipe through the different instructions (e.g.,using swipe gestures) to view the route segments between maneuvers.Here, the complex juncture indication is used, showing the intersectioncontext (a T intersection) and the maneuver made through theintersection, with the maneuver arrow emphasized over the intersectioncontext.

The fifth scenario 4225 illustrates navigation instructions in thecontext of the standard turn-by-turn navigation (i.e., not in thelock-screen mode, or with a different application open, etc.). In thiscase, the more complex arrow graphical indicator is used. In theillustrated example, the road juncture is slightly more complicated thanthe previous example, with a fourth branch angling up and to the rightfrom the direction of approach. The sixth scenario 4230 also illustratesnavigation instructions during turn-by-turn navigation. In this case,the maneuver being performed is a U-turn. Representing a U-turn with thejuncture branches as in scenario 4225 would result in the arrow pointingup and down the same branch (the bottom branch). As a result, theapplication instead displays a stored U-turn indicator arrow.

FIG. 43 illustrates several scenarios for the same turn, and how thedifferent arrows might be used for the same turn. The first scenario4305 shows a right turn onto 1^(st) St. in the turn-by-turn navigationinstructions. As in FIG. 42, the complex graphical indicator is used.The second scenario 4310 illustrates the situation during turn-by-turnnavigation in which the right turn onto 1^(st) St. is the second of twomaneuvers in quick succession. In this case, the second instructioncomes shortly after the first, so the application provides an indicationof the upcoming two maneuvers. The second maneuver is allotted lessspace on the display, and therefore the simplified arrow is used. Thethird scenario 4315 illustrates the use of the simplified arrowindicator in the route directions list. In addition, as shown for thesecond maneuver in the route directions list, some embodiments replacethe simplified directional indicator with a highway sign (shield) whenthe maneuver ends on a road for which such a shield/sign is available.The fourth and fifth scenarios 4320 and 4325 illustrate the simplifiedarrow indicators for the right turn in the lock-screen mode and when themapping application is not displayed on the device.

2. Download of Juncture Data and Generation of Graphical Indicators

In some embodiments, the mapping application performs navigation basedon a route selected by a user of the mapping application. For example, auser might request the mapping application to search for a route from afirst location to a second location (e.g., from the user's house to aparticular restaurant). In some embodiments, the application sends therequest to a centralized mapping service (e.g., a set of servers runningback-end map and route generation processes) and receives a set of oneor more possible routes from the first location to the second location.The user then selects one of the routes to follow.

FIG. 44 conceptually illustrates a process 4400 of some embodiments fordisplaying graphical indicators during route inspection. In someembodiments, a user can view a list of directions for a route (e.g., byselecting a list view GUI button) or can step through the directions oneat a time (e.g., via swipe gestures) while also viewing the route on themap. In some embodiments, the process 4400 is performed by a mappingapplication that operates on a device (e.g., a mobile device such as asmart phone or touchpad).

As shown, the process 4400 of some embodiments begins by sending (at4410) a request for a route to a mapping service server. In someembodiments, the request comprises a starting location and an endinglocation, potentially with one or more intermediate locations. The userenters these locations into the mapping application GUI of someembodiments, and the application transmits a route request through adevice interface to the mapping service server. The operations of theserver to generate a route and navigation (juncture) instructions aredescribed above in subsection A of this Section.

The process 4400 then receives (at 4420) the route along with encodedjuncture data. In some embodiments, the mapping service transmits thejuncture data in an encoded format. This encoding may simply involveidentifying similar junctures and referencing these rather thanrepeating the same juncture information twice, or may involve additionalencoding. Other embodiments do not provide any encoding. Assuming thedata is encoded, the process decodes (at 4430) the encoded juncture datato arrive at juncture information for each maneuver along a route. Thisjuncture data, in some embodiments, consists of geometry informationthat identifies the branches of the juncture and the angles at whichthose branches enter/exit the juncture. Some embodiments also includemaneuver information along with the juncture information, that describesthe maneuver being made (e.g., as a right turn, U-turn, freeway offramp, etc.).

Next, the process generates (at 4440) directional indicators for all thejunctures along the route. Directional indicators are graphicalindicators of route maneuvers along the route. For example, a route mayinclude a right turn at a first juncture, no turn at a second juncture,and a slight left at a third juncture. In this example, the set of routeindicators may include a first graphical representation for the rightturn (e.g., an arrow pointing to the right), a second graphicalrepresentation indicating no turn (e.g., a straight arrow), and a thirdgraphical representation for the slight left maneuver (e.g., a diagonalarrow to the left). Some embodiments, however, do not generate graphicalrepresentations for junctures at which the route continues through in astraight path. In fact, some embodiments do not transmit juncture datafor these junctures from the mapping service server. On the other hand,some embodiments do transmit juncture data for each juncture along theroute, and in some such embodiments the mapping application generatesgraphical indicators for each such juncture. In some embodiments, thedirectional indicators are generated by the device using a process suchas the process 4600 described below by reference to FIG. 46. In someembodiments, the application generates at least two directionalindicators for each maneuver: a first, more complex indicator thatincludes contextual information about the juncture, and a second,simpler indicator that only displays the maneuver to be made.

The process then determines (at 4450) whether a request to display routeinstruction(s) has been received. As shown in the previous subsection, auser might step through the instructions one at a time, or request toview a list of such route instructions. When no request is received, theprocess transitions to 4480, to determine whether the route inspectionhas ended (e.g., because the user has cancelled a route, begunnavigation of the route, closed the mapping application, etc.). Thesetwo operations effectively function as a ‘wait’ state, where the processwaits until an event causing the display of route instructions isreceived.

When the application has received such a request, the process 4400analyzes (at 4460) the context for displaying the one or moredirectional indicator(s). In some embodiments, the context depends onseveral factors associated with clearly displaying the route maneuversrequired for navigating the route. For example, the context may be basedon the amount of space available to display the graphical indicator(e.g., due to the size of the device on which the route directions aredisplayed), the conditions under which the indicator will be displayed(e.g., whether the maneuver is a current or future route maneuver, inwhich particular modality of the mapping application the sign will bedisplayed, etc.).

After identifying the context for the route instructions, the process4400 displays (at 4470) the directional indicator(s) for the maneuver(s)based on the context. In some embodiments, the context for displaying aparticular directional indicator determines how the directionalindicator appears when displayed. The directional indicators, in someembodiments, come in different illustrative styles for differentcontexts. A static (or simple) illustrative style for directionalindicators merely describes the maneuver by general appearance (e.g., anarrow turning right to direct the user to turn right, or an arrowturning slightly left to direct the user to turn slightly left, etc.). Adynamic illustrative style, in contrast, adapts and stylizes directionalindicators to clearly illustrate important aspects of each maneuver.Such stylized directional indicators can also include additional linesto illustrate other roads at the juncture, and other informationassociated with the maneuver. Some embodiments, for example, use themore complex directional indicators for displaying the routeinstructions one maneuver at a time, and use the simpler directionalindicators for displaying a list view of all of the instructions atonce. The process then determines (at 4480) whether route inspection hasended, as described above. Once route inspection has ended, the processends.

In addition to displaying route instructions, the directional indicatorsare used in various contexts during turn-by-turn navigation. FIG. 45conceptually illustrates a process 4500 of some embodiments thatperforms navigation over such a route. In some embodiments, the process4500 is performed by a mapping application that operates on a device(e.g., a mobile device such as a smart phone or touchpad).

As shown, the process 4500 begins by determining (at 4510) whether theuser is navigating the route. That is, the application determineswhether the location of the user device (e.g., provided by the device'sGPS capability or other location tracking mechanism) is along the pathof the route, or has moved off of the route. When the user moves off ofthe route (e.g., because the user makes a different maneuver than thosespecified by the route, taking the location of the device off theroute), the mapping application requires an update to the route andjuncture data. Accordingly, if the device running the mappingapplication is no longer on route, the process requests (at 4520) newroute and junction data from the mapping service server. The processthen receives (at 4530) revised route and juncture data for alljunctures along the route. In some embodiments, the juncture data isdetermined by the mapping service server for each juncture along theroute. As described above, the juncture data may include the angles ofthe different branches of the juncture, normalized to the entrydirection, along with an indication of the exit branch of the juncture.In some embodiments, the juncture data is retrieved by the server from astorage having a set of known junctures and angles (e.g., all the publicroads in the United States). In some cases, the server generatesjuncture data from other sources (e.g., transportation agencies ofstates and municipalities, recent satellite photos illustrating newroads not previously stored, etc.). For route updates, some embodimentsof the mapping service only generate and transmit new junctureinformation for the changes to the route, and reference thealready-received data for junctures shared by the old and new routes. Insome embodiments, as described above by reference to FIG. 45, thejuncture data is encoded, in which case the application also decodesthis data to arrive at the geometric juncture description.

After receiving the downloaded juncture data, the process 4500 generates(at 4540) directional indicators for all the junctures along the route.Directional indicators are graphical indicators of route maneuvers alongthe route. For example, a route may include a right turn at a firstjuncture, no turn at a second juncture, and a slight left at a thirdjuncture. In this example, the set of route indicators may include afirst graphical representation for the right turn (e.g., an arrowpointing to the right), a second graphical representation indicating noturn (e.g., a straight arrow), and a third graphical representation forthe slight left maneuver (e.g., a diagonal arrow to the left). Someembodiments, however, do not generate graphical representations forjunctures at which the route continues through in a straight path. Infact, some embodiments do not transmit juncture data for these juncturesfrom the mapping service server. On the other hand, some embodiments dotransmit juncture data for each juncture along the route, and in somesuch embodiments the mapping application generates graphical indicatorsfor each such juncture. In some embodiments, the directional indicatorsare generated by the device using a process such as the process 4600described below by reference to FIG. 46.

After generating the set of graphical directional indicators for thejunctures of the route, the process 4500 returns to 4510 to againdetermine whether the user is navigating the new route. When the userdevice is still following the route, the process 4500 determines (at4550) whether to display a new navigation sign. When navigating a route,in some embodiments, each maneuver associated with a juncture isillustrated to the user as a sign (e.g., a green sign with an arrow andtextual information indicating the type of maneuver) as the junctureapproaches. When a new navigation sign is not required (e.g., becausethe maneuver indicated by a currently displayed sign has not yet beenperformed), the process 4500 transitions to 4580 to determine whetherthe navigation has ended. When navigation has ended, the process 4500ends. These two operations effectively function as a ‘wait’ state, inwhich the mapping application waits for an event requiring the displayof a new navigation sign or for the navigation to end (e.g., because theroute's ending location has been reached).

When an event occurs requiring the display of a new sign, the process4500 identifies (at 4560) the context for displaying the sign. In someembodiments, the context depends on several factors associated withclearly displaying the route maneuvers required for navigating theuser-selected path. For example, the context may be based on the amountof space available to display the sign (e.g., due to the size of thedevice on which the navigation instructions are displayed), theconditions under which the indicator will be displayed (e.g., whetherthe maneuver is a current or future route maneuver, in which particularmodality of the mapping application the sign will be displayed, etc.).

After identifying the context for the navigation sign, the process 4500displays (at 4570) the directional indicator for the maneuver based onthe context. In some embodiments, the context for displaying the signdetermines how the directional indicator appears when it is displayed onthe sign. The directional indicators, in some embodiments, come indifferent illustrative styles for different contexts. A static (orsimple) illustrative style for directional indicators merely describesthe maneuver by general appearance (e.g., an arrow turning right todirect the user to turn right, or an arrow turning slightly left todirect the user to turn slightly left, etc.). A dynamic illustrativestyle, in contrast, adapts and stylizes directional indicators toclearly illustrate important aspects of each maneuver. Such stylizeddirectional indicators can also include additional lines to illustrateother roads at the juncture, and other information associated with themaneuver.

After displaying the directional indicator, the process 4500 transitionsto 4580 to determine whether the navigation has ended. In someembodiments, the navigation ends when the user stops the mappingapplication or when the destination is reached. If the navigation hasended, the process 4500 ends. Otherwise, the process 4500 transitionsback to 4510 to determine whether the route navigation is still onroute, as described above.

In some embodiments, the mapping application simplifies the routenavigation instructions by generating graphical directional indicators(e.g., arrows) for maneuvers (e.g., directions to turn, continuetraveling straight, etc.) along a route. FIG. 46 conceptuallyillustrates a process 4600 that generates such graphical directionalindicators for the maneuvers of a route. In some embodiments, theprocess 4600 is performed by a mapping application that operates on adevice (e.g., a mobile device such as a smart phone or touchpad). Themapping application performs this process 4600, in some embodiments, atstage 4520 of the process 4500, after receiving juncture data for alljunctures along a route. In some embodiments, this juncture data isgenerated and provided by a set of servers at a mapping service.

As shown, the process 4600 of some embodiments begins by selecting (at4610) a route maneuver at a juncture. In some embodiments, the routemaneuver is selected from a set of route maneuvers associated with alist of junctures along a specified route. The junctures for the routeare sequentially ordered, in some embodiments, according to thespecified route. The data for each juncture includes a set of branchesat specific angles, with an entrance and exit branch specified for themaneuver (in some embodiments, the entrance branch is specified by arotation of the juncture angles such that the entrance branch is at aspecific angle).

After selecting the route maneuver, the process 4600 performs (at 4620)a process to simplify the juncture, if such simplification is needed andpossible. In some embodiments, the simplification process uses a set ofsnapping rules to fit juncture branches to specific angles (e.g.,snapping a branch with an angle of 101.3° to 100° or 90°). In someembodiments, the simplification process is performed according to theprocess 4700 of FIG. 47. Some embodiments simplify the juncture data byattempting to snap each of the branches to a multiple of a particularangle, while other embodiments only attempt to snap the exit branch to amultiple of the particular angle.

Next, the process 4600 determines (at 4630) whether the mappingapplication was able to simplify the juncture. If the juncture could notbe simplified, the process 4600 uses (at 4640) a default representationof the route maneuver (e.g., a graphical icon of the route maneuverbased on the maneuver type). On the other hand, if the application wasable to simply the juncture, then the process generates (at 4650) asimple directional indicator for the maneuver based on the simplifiedjuncture. In some embodiments, the simple directional indicator is ageometry without any styling or other features (e.g., a simple arrowindicating the direction of the maneuver). For example, the simpledirectional indicator may be an arrow pointing up and then directlyright for a right turn maneuver at a standard juncture between two roadsor may be an arrow pointing up and then diagonally up and to the rightfor a slight right turn. The process 4600 also generates (at 4660) acomplex directional indicator based on the simplified juncture data. Insome embodiments, the complex directional indicator is a stylizedgraphical directional indicator that includes reference featuresassociated with the route maneuver. For example, the complex directionalindicator may include an emphasized directional arrow representing themaneuver to make at the juncture, and one or more de-emphasized linesindicating other roads at the juncture. For the complex indicators, theapplication displays de-emphasized lines for the branches of thejuncture through which the user will neither enter nor exit, anddisplays the emphasized arrow starting at the entrance branch andfinishing along the exit branch.

After generating the simple and complex directional indicators, theprocess 4600 determines (at 4670) whether the juncture at which theroute maneuver is made is the last juncture on the route. If thejuncture is the last, the process 4600 ends. Otherwise, when additionalroute maneuvers at additional junctures remain, the process transitionsback to 4610 to select the route maneuver at the next juncture.

As mentioned above by reference to FIG. 46, the mapping application ofsome embodiments receives juncture data (which may have complexgeometry) and simplifies the juncture geometry for use in the graphicaluser interface (GUI). In some embodiments, the mapping applicationsimplifies the route navigation by fitting route juncture angles tomultiples of a pre-specified simplified angle. FIG. 47 conceptuallyillustrates a process 4700 that attempts to set the angles of branchesof a juncture along a route to multiples of a pre-specified angle (e.g.,45°). This process 4700 is performed, in some embodiments, at stage 4620of FIG. 46. The process 4700 will be described by reference to FIGS. 48and 49, which illustrate particular juncture situations.

As shown, the process 4700 begins by identifying (at 4710) the angles ofthe juncture through which a route maneuver is made. These angles, insome embodiments, are those specified by the mapping service server forthe juncture, and also indicate one of the branches as the exit branch.The top portion of FIG. 48 illustrates a first point juncture 4805 on amap, with angles at approximately −55°, 90°, and −115° (with North as0°). In addition, the map shows a maneuver through the juncture,resulting in a slight right turn from the −115° branch to the 90°branch. The second stage in this top portion shows the juncture andmaneuver reduced to geometry by the mapping service server, with thejuncture data rotated so that the maneuver entrance branch is at 180°.This shows the exit branch at 25° and the third branch at around −60°

In addition, the bottom portion of the figure illustrates a roundaboutjuncture 4810 on a map, in which the user enters on a first branch ataround −110° and exits on a second branch at around 150°, with a thirdbranch at 0°. Again, the second stage illustrates the juncture andmaneuver reduced to geometry by the mapping service server, includingrotation of the juncture so that the entrance branch is set to 180°. Inaddition, the juncture is marked as a roundabout.

After the angles for the juncture are identified, the process 4700identifies (at 4720) a possible modification to each of the angles ofthe juncture based on a set of snapping rules. In some embodiments, thesnapping rules indicate how to adapt a received angle for display duringnavigation. In some embodiments, the snapping rules indicate thatreceived angles should be modified to a multiple of a pre-specifiedangle. For example, the snapping rules may indicate that eachdirectional indicator should conform to one of several axes at multiplesof 45 degrees (e.g., 45°, 90°, 135°, 180°). In some embodiments, thesnapping rules specify that the received angle should be adapted to the45° axis closest to the received angle. For example, the received angleis snapped to 90° for a right-turn maneuver onto a road that is at a110° angle with respect to a reference angle (i.e., the direction oftravel into the juncture) because the closest axis among the 45°multiples is the axis at 90°. On the other hand, the angle direction issnapped to 135° for the right-turn maneuver if the road is at a 115°angle with respect to the reference point because the 115° angle of thejuncture branch is closer to the 135° axis than to the 90° axis.

After identifying the possible modifications to the angles of theselected juncture, the process 4700 determines (at 4730) whether theidentified modification is inconsistent with the type of maneuverassociated with the received angle. For instance, if a maneuver involvesa slight right turn at an angle of 10°, and the snapping rules modifythe 10° branch angle to a 0° angle (moving straight through thejuncture), then a simplified directional indicator for the maneuverwould merely illustrate a graphical representation for travelingstraight. Lost in such an indicator would be any indication of turning,veering, or moving to the right. In this example, moving straight isinconsistent with turning right (even slightly turning right).

When the identified modification is inconsistent with the maneuver type,then the process searches (at 4740) for other modifications to the angleof the exit branch of the juncture. In some embodiments, the snappingrules specify alternative angles to use when a first angle isinconsistent. For example, the snapping rules may indicate that a 45°angle should be used after the 0° angle was determined to beinconsistent the right turn maneuver.

Next, the process determines (at 4750) whether an acceptablemodification was found. In some embodiments, an alternative angle isacceptable if the difference between the received angle for the exitbranch (e.g., 10° angle) and the identified alternative angle (e.g., 45°angle) is within a threshold. For example, the snapping rules mayspecify a maximum difference of 30° as the threshold for an acceptablealternative angle. In this example, the identified 45° alternative angleis 35° greater than the 10° received angle, and therefore, would beconsidered unacceptable. In some embodiments, the determination is madebased on one or more heuristic rules that consider the context formodifying the angle. For example, a heuristic rule may specify that aright turn should always be shown when there is a fork in the road. Inthis example, even a slight turn to the right (e.g., at an angle of 10°)could be illustrated by an alternative directional indicator (e.g., at a45° angle).

When no acceptable modification is found, the process determines andspecifies (at 4760) that no simplification can be made to the geometryof the juncture, for at least one of the branches. As indicated above byreference to the process 4600, some embodiments use defaultrepresentations for the maneuver when no simplification can be made tothe juncture geometry. The process then ends.

When an acceptable modification is found for the exit branch (either inthe initial determination at 4730 or the secondary determination at4750), the process transitions to 4770 to determine whether themodification results in the overlap of two branches of the juncture(i.e., two branches assigned to the same angle). When two (or more) ofthe branches of the juncture overlap after the modifications, theprocess determines and specifies (at 4760) that the modifications shouldnot be made. On the other hand, when there is no overlap betweenbranches and the resulting exit branch is not inconsistent with themaneuver, then the process sets (at 4780) the angles of the juncturebranches to the angles determined for the modified juncture. The processthen ends.

The third stage in each of the portions of FIG. 48 illustrates thesimplified geometry for the two example intersections. In the case ofthe point juncture 4805, the application simplifies the geometry toalign the 25° branch to 45° and the −60° branch to −45°. As thesealignments do not create any issues (e.g., confusing instructions oroverlapping branches), the mapping application uses the simplifiedgeometry to generate a directional indicator for the juncture andmaneuver. In the case of the roundabout juncture 4810, the applicationsimplifies the geometry to align the 80° branch to 90° and the −70°branch to −90°. As these alignments also do not create any issues, themapping application uses the simplified geometry to generate aroundabout directional indicator for the juncture and maneuver. In someembodiments, however, the simplification process is not applied toroundabout junctures. In many cases, the user's (i.e., the driver's)perspective changes over the course of a roundabout maneuver, andsometimes the roundabout is large enough that the user cannot initiallysee the exit road. Instead, the directional indicator uses angles thatmore closely resemble the actual exits of the roundabout.

FIG. 49 illustrates two examples where default juncture/maneuverindicators are used instead of the geometry-based indicators. The firstsituation is a U-turn maneuver at a juncture 4905. In this case, thejuncture data stored for the juncture is a standard juncture with fourbranches in the cardinal directions. However, the data for the maneuverincludes the exit branch at 180°, which is also the entrance branch.Thus, while no simplification is necessary of the geometry, thedirectional indicator that would be generated according to the standardrules would just be an arrow pointing downward, overlapping the entranceportion of the arrow. As this would not be a very useful graphicalindication of the maneuver, the mapping application instead uses astored U-turn indicator, as shown in the fourth stage of the top portionof this figure.

The second example in FIG. 49 is a freeway exit 4910. In this case, thegeometry for the juncture and maneuver includes the entrance branch, abranch at 0° (for continuing straight along the freeway) and a branch ata low angle (around 10°). The attempt at simplifying the geometry,however, reduces the exit branch to 0°. This violates multiplesimplification rules, in that it results in overlapping juncturebranches and creates a non-intuitive description of the maneuver,because the exit branch now indicates that the route should continue ina straight line. While some embodiments will instead shift the exitbranch to 45°, this example instead uses a default freeway exitgraphical indicator. In some embodiments, as mentioned above, eachjuncture also includes routing directions, such as “Freeway_Off_Ramp”.In this case, the directions stating that the maneuver involves taking afreeway off ramp, combined with the knowledge that the route is in aright-sided driving area and that the exit branch is at a low angle tothe right of straight through, indicates that the right-side freeway offramp graphical indicator should be used. Other examples of maneuvertypes that may use default representations include, in some embodiments,“keep left” or “keep right” maneuvers.

FIG. 50 illustrates an example of a roundabout 5010 at which thesimplified geometry is not used by some embodiments. In this case, thegeometry received for the roundabout includes the entrance branch, abranch at approximately 110°, a branch at 85°, and the exit branch atapproximately −70°. The attempted simplification places both of theunused (neither entrance nor exit) branches on the 90° axis. For a pointintersection, some embodiments might allow this for the maneuver, as theinstructions to make a left turn would be clear. However, when a drivergoes around a roundabout, they will often count the number of exits, asthe angles become unclear as perspective changes. Thus, reducing thenumber of branches could be potentially confusing for the driver. Assuch, the directional indicator used eliminates the exits altogether andonly shows a circular roundabout with an arrow showing the maneuver. Inaddition, some embodiments include accompanying instructions that state“Take the third exit of the roundabout”, or other instructions to thateffect.

In addition, certain situations will result in the applicationeliminating the directional indicator from a navigation sign altogetherdue to the possibility of confusing the user. For instance, anintersection involving an interchange from a first freeway to a secondfreeway might have three lanes that go in two or three differentdirections. In certain cases, the information generated on the serverdoes not specify which lane the user should take in order to get to thedesired second freeway (i.e., whether to use the left lane or the rightlane). While the application of some embodiments could show a genericarrow for such a maneuver, this might confuse the user. Accordingly, thenavigation application of some embodiments suppresses the graphicaldirectional indicator and centers the instruction text in the navigationsign. When a highway shield is available for the second (destination)freeway, some embodiments move the instructions to the left side of thesign and display the highway shield on the right side of the sign.

3. Direction Indicator Software Architecture

As stated above, in some embodiments the maps, routes, and turn-by-turnnavigation are presented to a user by a mapping application thatoperates on a device (e.g., a handheld device such as a smart phone ortablet). The mapping application may be a stand-alone application insome embodiments, or integrated with the operating system of the device.FIG. 51 conceptually illustrates a mapping application 5100 of someembodiments that generates directional indicators for differentcontexts. One of ordinary skill in the art will recognize that themodules shown for the application 5100 are specific to the arrowgeneration processes, and that the mapping application of someembodiments includes numerous additional modules (e.g., for map display,route display, additional aspects of navigation, etc.).

As shown, a mapping service server 5110 transmits route and juncturedata through a network 5115 to a network interface 5120 of the device onwhich the mapping application 5100 operates. The mapping service server5110 may be a server similar to that shown in FIG. 24 above, whichreceives a route request from devices on which the mapping applicationoperates and generates route and juncture data for the requests.

The mapping application 5100 includes a juncture decoder 5130, ageometry simplifier 5145, an arrow generator 5160, an arrow selector5165, a context analyzer 5175, and a sign generator 5180. The juncturedecoder 5130 receives encoded juncture information 5125 for a route anddecodes this information to arrive at a series of maneuvers throughjunctures. The juncture decoder 5130 stores the decoded juncture data5135. This may be in random access memory or other volatile storage, foruse only during the navigation of the route, or in a more permanentstorage such as a hard disk or solid-state memory. As stated above, someembodiments do not encode the juncture information, in which case theapplication does not require a juncture decoder, and simply stores thereceived juncture data.

The juncture data 5135, in some embodiments, includes a geometricdescription of the intersection that indicates the type of theintersection (e.g., point, roundabout) as well as the different branchesof the intersection at their angles, according to an analysis of the mapby the mapping service. As the junctures correspond to maneuvers to makealong a particular route, the juncture data also indicates an exitbranch for each juncture. This figure illustrates an example geometricjuncture description 5140, with three branches and the exit marked withan arrow. The branch at 180° (the bottom branch) is always assumed to bethe entrance branch, in some embodiments.

The geometry simplifier 5145 reduces the juncture data to a simplifiedform and stores simplified juncture data 5155. As with the decodedjuncture data 5135, this data may be stored in volatile or non-volatilememory in different embodiments. In some embodiments, the geometrysimplifier attempts to snap each of the branches of a juncture to amultiple of 45°, according to various heuristic rules. The geometricjuncture description 5150 is a simplified version of the geometricdescription 5140.

The arrow generator 5160 generates one or more graphical indicators foreach juncture/maneuver, using the simplified juncture data. When theindicator generated according to the juncture data is not ideal (e.g.,for a U-turn, a freeway exit maneuver, etc.), the arrow generator ofsome embodiments uses the stored default indicators 5170. For at leastsome of the junctures, the arrow generator creates a complex directionalindicator (that also includes a de-emphasized representation of thejuncture) and a simple directional indicator. The arrow generator 5160stores these directional indicators 5162 for use in displaying route andnavigation instructions, in either volatile or non-volatile memory.

The arrow selector 5165 uses a context analyzer 5175 to determine whichof the directional indicators to use for a particular maneuver,depending on the context in which the indicator will be displayed. Thesecontexts may include different situations for routing directions ordifferent situations for turn-by-turn navigation instructions (e.g.,standard mode, lock-screen mode, when a different application is open,etc.). The context analyzer 5175 identifies the context and providesthis information to the arrow selector 5165.

The arrow selector chooses one of the graphical indicators 5162 andprovides this selection to the sign generator 5180. The sign generator5180 generates a navigation instruction sign for display that includesthe selected graphical indicator. The sign generator 5180 also uses thecontext analyzer results to generate other aspects of the sign, such asthe level of detail of the instructions shown within the navigationsign.

IV. Dynamic Generation of Adaptive Instructions

As shown in many of the figures in the above sections, in addition todisplaying a graphical indication of a maneuver during a route, themapping application of some embodiments displays maneuver instructions(e.g., “Turn left in 0.5 miles onto Bahrami Ct.”). Much like thegraphical indicators, the mapping application dynamically generatesthese instructions using the received route/junction data.

A. Examples of Different Instructions for Same Maneuver in DifferentContexts

The mapping application of some embodiments displays textual routeinstructions in a large variety of cases, some of which are more spaceconstrained than others, and some in which other guidance elementsprovide information about a maneuver that can take the place of the textinstructions. Rather than selecting a single instruction string and thenshrinking the font or truncating as dictated by the constraints, theapplication uses a sophisticated method to synthesize strings that arebest adapted to each context from a number of details about the maneuveritself.

For a given context, the application chooses instruction text byconsidering factors such as the available space, the amount ofinformation conveyed by means other than text (e.g., the graphicalindicators, road signs, etc.), the localized length of each of theinstruction variants, among other factors. By synthesizing andevaluating several alternatives locally on the client device (as opposedto simply receiving instruction text from the mapping service), themapping application can pick an optimal instruction string in everyscenario. In addition, this approach allows for the application to usedifferent instruction text on a differently-sized device (e.g., usingmore text on a tablet computer as compared to a smaller smart phone). Asimilar approach can also be used for spoken instructions that need tofit within a particular amount of time, and when voice instructions areused, the application of some embodiments will reduce the length of thedisplayed instructions.

FIG. 52 illustrates an example of the synthesis of differentinstructions for a particular maneuver at a juncture according to someembodiments. FIGS. 53 and 54 then illustrate different scenarios inwhich these different instructions for the maneuver are used. As shown,the mapping application uses received route instructions and juncturedata to identify specific aspects of maneuver instructions. The table5205 conceptually illustrates how various strings might be generated fora juncture. Specifically, the maneuver instructions include an “At”field, a “Turn” field, an “Onto” field, a “Towards” field, and a “For”field. For each juncture, the application initially populates thesestring fields, in order to synthesize the instructions from the fields.

In some embodiments, the “At” field is based on map information thatincludes traffic light and stop sign information, etc. For the examplesshown in FIG. 52, the first juncture takes place “at the end of theroad”, while the second juncture takes place “at the next light”. The“Turn” field describes the maneuver to be made; examples of this fieldinclude “turn right” (the maneuver performed at the first juncture),“exit freeway”, “keep left”, “slight left turn”, “U-turn”, or othermaneuvers. The route directions that include a maneuver description maybe mapped to different possible strings for the “Turn” field.

The “Onto” field indicates the pathway (i.e., street, freeway, etc.)onto which the maneuver exits the juncture. In the case of the firstjuncture in FIG. 52, the maneuver exits the juncture “onto 1^(st) St.”.The “Towards” field indicates a marker (taken from the map data orjuncture data) towards which the exit branch points. In someembodiments, the mapping application analyzes the exit branch of thesubsequent juncture, and uses the name of this road as the “towards”field. In the example, the second juncture is a left turn onto B St., sothe “Towards” field for the first juncture indicates that the maneuverexits “towards B St.” Other embodiments use either the next road withwhich the exit street of the present junction intersects, a major road(e.g., a freeway), or other easily recognizable descriptor (e.g., acity, etc.). The “For” field indicates the distance along which theroute will follow the road in the “Onto” field (that is, the road ontowhich the juncture exits). Thus, in the example instructions, the nextjuncture will be in 0.1 miles, so the “For” field is “for 0.1 miles”.

Next, after generating each of the component strings for a set ofinstructions, the mapping application of some embodiments generatesdifferent levels of instructions. The table 5200 illustrates a set ofsynthesized instructions for the first juncture. Specifically, the table5200 illustrates five sets of instructions, of varying lengths, for aparticular juncture. However, one of ordinary skill in the art willrecognize that different embodiments might include fewer, additional, ordifferent synthesized strings based on the set of string fields.

The first instruction set uses all five of the fields. This is thelongest instruction set, reading “At the end of the road, turn rightonto 1^(st) St., towards B. St. for 0.1 miles”. As it is the longestinstruction set, the application assigns the instruction set a rankof 1. The second instruction set removes the “For” field, using only the“At”, “Turn”, “Onto”, and “Towards” fields. The third instruction setremoves the “At” field. These fields add context, and are therefore niceto have when additional room is available. However, they are lessintegral to the maneuver itself, and therefore are the first fields toremove when shortening the instruction text. Next, for the fourthinstruction set, the application removes the “Towards” field, as the“Turn” and “Onto” fields are considered more important. Lastly, thefifth instruction set contains only the “Turn” field, simply stating“Turn right”.

Again, some embodiments will include additional instruction sets, whendifferent length instructions (that still make sense) are available. Forinstance, some embodiments will include an instruction set that removesthe “At” field but keeps the “For” field, in the case that the “For”field is shorter than the “At” field. This enables the application tohave another option in case the second instruction set (with the “For”field removed) is just slightly too long for the allocated space.Furthermore, some embodiments may include additional, fewer, ordifferent fields. For instance, some embodiments might include a “In”field, that gives the distance to the upcoming juncture (i.e., “In 0.5miles, . . . ”).

FIGS. 53 and 54 illustrate several different scenarios in which themapping application displays different examples of the adaptiveinstructions for the particular maneuver of the first juncture in table5205 in a variety of different situations. In this case, the fullinstructions are “In 0.5 miles, at the end of the road, turn right onto1^(st) St. towards B. St. for 0.1 miles.” However, as the example doesnot include an “In” field, the highest ranked instructions are slightlyshorter than this. In order to determine which instruction set to usefor a particular display, the mapping application of some embodimentsdetermines a maximum length for the instruction set, then chooses thehighest ranked set that fits into the allotted space.

The first scenario 5305 illustrates instructions for the particularmaneuver displayed during turn-by-turn navigation. In this case, theapplication allots three text lines for the instruction. The distance(0.5 miles) is already displayed in large font at the top of thenavigation sign, but this is not counted as one of the text lines. Withthree lines available, the highest ranked instruction set can be used inthe navigation sign.

The second scenario 5310 illustrates turn-by-turn navigationinstructions for the particular maneuver while in lock screen mode. Inthis mode, only two lines of large text are allotted in someembodiments, so the highest ranked instructions that fit use only the“Turn” and “Onto” fields. This simplifies into the direction of the turnand the street onto which the user turns. The third scenario 5315illustrates navigation instructions for the maneuver while the mappingapplication is not open on the device, in which case the instructionsshow up as an alert banner. In this case, the application only allotsone line to the instructions, so the lowest ranked instructions (“Turnright”) are used.

The fourth scenario 5320 illustrates the display of information in thelist view for route directions. This view, as described above, listssubsequent instructions for each of the maneuvers along a route. In someembodiments, the banners in the list view for each direction are of avariable height, and therefore the full instruction set is always used.Thus, the highest ranked set of instructions, “At the end of the road,turn right onto 1^(st) St. towards B. St.” is used for the firstmaneuver in the list. As shown, this maneuver takes an extra line oftext as compared to the next two maneuvers.

The fifth scenario 5325 illustrates turn-by-turn navigation in 3D mode.As compared to the first scenario 5305, some embodiments allot less roomin the navigation sign for the instruction set when in 3D mode, in orderfor more of the 3D display to be viewable. As such, the application usesthe third ranked instruction set, because this is the largestinstruction that fits in the two lines using the given text size.

FIG. 54 illustrates additional scenarios in which the mappingapplication uses the synthesized instruction sets. The sixth scenario5405 illustrates the display of route overview instructions that theuser can step through (e.g., with sweep gestures). In some embodiments,the application allots the same amount of space for step-throughinstructions as for turn-by-turn navigation, and therefore theapplication again uses the highest ranked instruction set that includesall of the fields.

The seventh scenario 5410 is the same as the first scenario 5305, butexplicitly indicates that the spoken navigation is turned off. This isprovided here to contrast with the eighth scenario 5415, in which voiceinstructions are enabled during turn-by-turn navigation. For voicenavigation, the application determines a maximum amount of time allowedfor speaking the instructions, then determines the highest ranked set ofinstructions that can be spoken within this allotted time. In this case,the time allows the entirety of the highest ranked instruction set to beselected. In addition, when voice navigation is activated, theapplication reduces the size of the displayed navigation sign. As such,the application displays the third ranked instruction set within thedisplay.

Finally, the mapping application of some embodiments may operate ondifferent types of devices with different size display screens. Forexample, the application might operate on both smart phones and largertablet computers. When operating on a larger device, some embodimentsallow more room for the navigation sign. The ninth scenario 5420illustrates turn-by-turn 3D navigation on a larger device (a tabletcomputer). Unlike in the fifth scenario 5325, the navigation signprovides enough room for the highest ranked instruction set to be used.

The above description describes some embodiments that generate severaldifferent instruction sets for a maneuver, rank the instruction sets,and then adaptively determine which of these instruction sets best fitsinto a particular space. In some embodiments, the application identifiesa maximum number of characters available to use for the instructiondisplay. The application then starts with the highest ranked instructionset and determines whether the instruction set fits into the identifiednumber of characters. When the instruction set fits, the applicationselects and displays the instruction set. When the instruction set doesnot fit, the application moves to the next ranked instruction set andperforms the same test. If none of the instruction sets fit, then theapplication uses the one that comes closest to fitting. Some embodimentsthen truncate the instruction set with an ellipsis to indicate that theinstruction set does not completely fit within the space. This mayresult in elements being removed from the string.

In addition to text, some embodiments use text substitutes within theinstruction sets. Specifically, for roads represented by shield signs(e.g., interstate freeways, state routes), the application uses theshield representation of the road rather than the road name (e.g., ablue and red shield with “I-5” inside of it instead of “Golden StateFreeway” or “Interstate 5”. Some embodiments treat these signs as afixed number of characters when assessing the different instructionsets.

The above description describes some embodiments of the mappingapplication in which the decision regarding which elements to use isperformed primarily based on trying to use the maximum lengthinstruction set. Some other embodiments factor in whether certainelements of an instruction set are presented to the user in a differentvisual manner, and may potentially remove these elements.

For instance, when displaying a detailed instructional arrow that makesclear a turn is a slight right turn, some embodiments shorten theinstruction to remove the “slight” or even remove the entire referenceto the turn, instead using instructions along the line of “CA-17 Stowards Santa Cruz”. Similarly, if displaying a large road shield sign,then the “CA-17 S” portion of the instruction might be omitted.

B. Client Side Generation of Instructions that are Adapted to theContext

The above section illustrated several examples of the mappingapplication synthesizing navigation instructions based on route andjuncture data, and then displaying different variants of theseinstructions according to different contexts. A user might request themapping application to search for a route from a first location to asecond location (e.g., from the user's house to a particularrestaurant). In some embodiments, the application sends the request to acentralized mapping service (e.g., a set of servers running back-end mapand route generation processes, such as those described above in SectionIII.A) and receives a set of one or more possible routes from the firstlocation to the second location. The user then selects one of the routesto follow.

FIG. 55 conceptually illustrates a process 5500 of some embodiments fordisplaying text instructions during route inspection. In someembodiments, a user can view a list of directions for a route (e.g., byselecting a list view GUI button) or can step through the directions oneat a time (e.g., via swipe gestures) while also viewing the route on themap. In some embodiments, the process 5500 is performed by a mappingapplication that operates on a device (e.g., a mobile device such as asmart phone or touchpad).

As shown, the process 5500 of some embodiments begins by sending (at5510) a request for a route to a mapping service server. In someembodiments, the request comprises a starting location and an endinglocation, potentially with one or more intermediate locations. The userenters these locations into the mapping application GUI of someembodiments, and the application transmits a route request through adevice interface to the mapping service server. The operations of theserver to generate a route and navigation (juncture) instructions aredescribed above in Section III.A.

The process 5500 then receives (at 5520) the route along with encodedjuncture data. In some embodiments, the mapping service transmits thejuncture data in an encoded format. This encoding may simply involveidentifying similar junctures and referencing these rather thanrepeating the same juncture information twice, or may involve additionalencoding. Other embodiments do not provide any encoding. Assuming thedata is encoded, the process decodes (at 5530) the encoded juncture datato arrive at juncture information for each maneuver along a route. Thisjuncture data, in some embodiments, consists of geometry informationthat identifies the branches of the juncture and the angles at whichthose branches enter/exit the juncture. Along with the junctureinformation, some embodiments also include maneuver information thatdescribes the maneuver being made (e.g., as a right turn, U-turn,freeway off ramp, etc.).

Next, the process generates (at 5540) text instruction variants for allthe junctures along the route. Text instruction variants arecombinations of text strings derived from the decoded juncture andmaneuver information. As discussed above in conjunction with FIGS.52-54, examples of such text strings include “at the secondintersection”, “turn left”, “onto 1^(st) St.”, “towards Wolfe Rd.”, and“for 0.3 miles”. In some embodiments, the process 5500 combines the textstrings into text instruction variants. As a first example of such acombination, process 5500 may combine “at the second intersection” and“turn left” to produce a short text instruction variant that reads, “Atthe second intersection, turn left.” As a second example of such acombination, process 5500 may combine all of the previous text stringsto produce a long text instruction variant that reads “At the secondintersection, turn left onto 1^(st) St., towards Wolfe Rd. for 0.3Miles.” In some embodiments, process 5500 ranks the text instructionvariants for each juncture based on the amount of information conveyedin each variant. In some embodiments, the text instruction variants aregenerated by the device using a process such as the process 5800described below by reference to FIG. 58.

The process then determines (at 5550) whether a request to display routeinstruction(s) has been received. As shown in the previous subsection, auser might step through the instructions one at a time, or request toview a list of such route instructions. When no request is received, theprocess transitions to 5580, to determine whether the route inspectionhas ended (e.g., because the user has cancelled a route, begunnavigation of the route, closed the mapping application, etc.). Thesetwo operations effectively function as a ‘wait’ state, where the processwaits until an event causing the display of route instructions isreceived.

When the application has received such a request, the process 5500analyzes (at 5560) the context for displaying the one or more textinstruction(s). In some embodiments, the context depends on severalfactors associated with clearly displaying the instructions required fornavigating the route. For example, the context may be based on theamount of space available to display the text instruction (e.g., due tothe size of the device on which the route directions are displayed) orthe conditions under which the indicator will be displayed (e.g.,whether the maneuver is a current or future route maneuver, in whichparticular modality of the mapping application the sign will bedisplayed, etc.).

After identifying the context for the route instructions, the process5500 displays (at 5570) the text instruction(s) for the maneuver(s)based on the context. In some embodiments, the context for displaying aparticular text instruction determines which text instruction variant isdisplayed. The text instruction variants, in some embodiments, come indifferent lengths for different contexts. In some embodiments, longertext instruction variants convey more information than shorterinstruction variants. However, longer instruction variants may not fitinto small banners or may cause wrapping effects across text lines. Someembodiments, for example, use longer text instruction variants fordisplaying the route instructions one maneuver at a time in the standardturn-by-turn navigation view, but use shorter text instruction variantsfor displaying the same maneuver when less space is allocated, such aswhen navigation is on but the device is in a different application. Theprocess then determines (at 5580) whether route inspection has ended, asdescribed above. Once route inspection has ended, the process ends.

In addition to the display of route instructions before actuallyfollowing along a route, the text instructions are used in variouscontexts during turn-by-turn navigation. FIG. 56 conceptuallyillustrates a process 5600 of some embodiments that performs navigationover such a route. In some embodiments, the process 5600 is performed bya mapping application that operates on a device (e.g., a mobile devicesuch as a smart phone or touchpad).

As shown, the process 5600 begins by determining (at 5610) whether theuser is navigating the route. That is, the application determineswhether the location of the user device (e.g., provided by the device'sGPS capability or other location tracking mechanism) is along the pathof the route, or has moved off of the route. When the user moves off ofthe route (e.g., because the user makes a different maneuver than thosespecified by the route, taking the location of the device off theroute), the mapping application requires an update to the route andjuncture data. Accordingly, if the device running the mappingapplication is no longer on route, the process requests (at 5620) newroute and junction data from the mapping service server. The processthen receives (at 5630) revised route and juncture data for alljunctures along the route. In some embodiments, the juncture data isdetermined by the mapping service server for each juncture along theroute. As described above, the juncture data may include the angles ofthe different branches of the juncture, normalized to the entrydirection, along with an indication of the exit branch of the juncture.In some embodiments, the juncture data is retrieved by the server from astorage having a set of known junctures and angles (e.g., all the publicroads in the United States). In some cases, the server generatesjuncture data from other sources (e.g., transportation agencies ofstates and municipalities, recent satellite photos illustrating newroads not previously stored, etc.). For route updates, some embodimentsof the mapping service only generate and transmit new junctureinformation for the changes to the route, and reference thealready-received data for junctures shared by the old and new routes.

Next, process 5600 decodes (at 5640) the encoded juncture data to arriveat juncture information for each maneuver along the revised route. Thisjuncture data, in some embodiments, consists of geometry informationthat identifies the branches of the juncture and the angles at whichthose branches enter/exit the juncture. Along with the junctureinformation, some embodiments also include maneuver information thatdescribes the maneuver being made (e.g., as a right turn, U-turn,freeway off ramp, etc.).

The process then generates (at 5650) text instruction variants for thejunctures in the revised route. Text instruction variants arecombinations of text strings derived from the decoded juncture andmaneuver information. As discussed above in conjunction with FIGS.52-54, examples of such text strings include “at the secondintersection”, “turn left”, “onto 1^(st) St.”, “towards Wolfe Rd.”, and“for 0.3 miles”. In some embodiments, the process 5500 combines the textstrings into text instruction variants. As a first example of such acombination, process 5500 may combine “at the second intersection” and“turn left” to produce a short text instruction variant that reads, “Atthe second intersection, turn left.” As a second example of such acombination, process 5500 may combine all of the previous text stringsto produce a long text instruction variant that reads “At the secondintersection, turn left onto 1^(st) St., towards Wolfe Rd. for 0.3Miles.” In some embodiments, process 5500 ranks the text instructionvariants for each juncture based on the amount of information conveyedin each variant. In some embodiments, the text instruction variants aregenerated by the device using a process such as the process 5800described below by reference to FIG. 58.

After generating the set of text instruction variants for the juncturesin the revised route, the process returns to 5610 to again determinewhether the user is navigating the new route. When the user device isstill following the route, the process 5600 determines (at 5660) whetherto display a new navigation instruction. When navigating a route, insome embodiments, each maneuver associated with a juncture isillustrated to the user as a sign (e.g., a green sign with an arrow anda textual instruction describing the maneuver at some level of detail)as the juncture approaches.

When a new navigation instruction is not required (e.g., because themaneuver indicated by the currently displayed instruction has not yetbeen performed), the process determines (at 5675) whether to update thecurrently displayed instruction. As shown in the previous subsection, insome embodiments the navigation sign includes an indicator of thedistance remaining until the maneuver. In some embodiments, the mappingapplication regularly updates this distance indicator as the maneuverapproaches (e.g., every mile, then every tenth of a mile, then everyfifty feet, etc.). When a distance threshold has been reached such thatthe displayed instruction must be updated, the process 5600 updates (at5680) the displayed instruction. In some embodiments, this entailsupdating the number and/or distance unit (e.g., switching from miles tofeet for the last tenth of a mile). Some embodiments also provide anupdate to the distance remaining via the voice output feature of thedevice on which the process operates.

The process then proceeds to 5685 to determine whether navigation hasended. When navigation has ended, the process 5600 ends. The operations5660, 5675, and 5685 effectively function together as a ‘wait’ state, inwhich the mapping application waits for an event requiring either thedisplay of a new navigation instruction or an update to the displayednavigation instruction, or for the navigation to end (e.g., because theroute's ending location is reached).

When an event occurs requiring the display of a new navigationinstruction, the process analyzes (at 5665) the context for displayingthe text instruction. In some embodiments, the context depends onseveral factors associated with clearly displaying the route maneuversrequired for navigating the route. For example, the context may be basedon the amount of space available to display the text instruction (e.g.,due to the size of the device on which the route directions aredisplayed) or the conditions under which the indicator will be displayed(e.g., whether the maneuver is a current or future route maneuver, inwhich particular modality of the mapping application the sign will bedisplayed, etc.). Additional contextual factors may include whether theinstruction is also being provided to the user via the voice guidancefeature, the orientation of the device, and other factors in someembodiments.

After identifying the context for displaying the text instructions, theprocess 5600 displays (at 5670) a text instruction variant for theupcoming juncture based on the context. In some embodiments, the contextfor displaying a particular text instruction determines which textinstruction variant is displayed. The text instruction variants, in someembodiments, come in different lengths for different contexts. In someembodiments, longer text instruction variants convey more informationthan shorter instruction variants. However, longer instruction variantsmay not fit into small banners or may cause wrapping effects across textlines. Some embodiments, for example, use longer text instructionvariants for displaying the route instructions one maneuver at a time,and use shorter text instruction variants for displaying a list view ofall of the instructions at once. Some embodiments set a character orsize limit based on context, and select the text instruction variantthat conveys the most information while still fitting into the displayarea allotted for the navigation instruction and maintaining theappropriate text size.

After displaying the new instructions, the process 5600 transitions to5685 to determine whether the navigation has ended. In some embodiments,the navigation ends when the user stops the mapping application or whenthe destination is reached. If the navigation has ended, the process5600 ends. Otherwise, the process 5600 transitions back to 5610 todetermine whether the route navigation is still on the path, asdescribed above.

As discussed above, the mapping application of some embodiments decodesand synthesizes the received juncture data in order to arrive at thestring data with which to generate the navigation instructions. FIG. 57conceptually illustrates a process 5700 of some embodiments for decodingencoded juncture data and synthesizing instruction elements from theroute and juncture data received from a mapping service. In someembodiments, the process 5700 is performed by a mapping application. Themapping application performs this process 5700, in some embodiments, atstage 5530 of the process 5500, after receiving juncture data for alljunctures along a route.

As shown, the process 5700 begins by receiving (at 5710) encodedjuncture data for all junctures in the route. In some embodiments, theprocess downloads the encoded juncture data from a mapping serviceserver. In some embodiments, this encoding may simply involveidentifying similar junctures and referencing these within the set ofjunctures rather than repeating the same juncture information twice, ormay involve additional encoding. In addition, some embodiments receiveroute instruction information that indicates a type of maneuver toperform (e.g., “turn right”, “keep left”, etc.).

Next the process selects (at 5720) an unprocessed juncture. In someembodiments, the process selects the junctures in order from the startof the route to the end of the route, at each maneuver indicated by theroute instructions. Once a juncture is selected, the process analyzes(at 5730) any relationships the selected juncture has with preceding orsubsequent junctures, as well as specific aspects of the selectedjuncture. In some embodiments, such analysis can include determiningdistances between the preceding and subsequent junctures in the route,determining turn degree, identifying the primary maneuver to perform atthe selected juncture, identifying the names of the roads at each branchof the juncture, identifying roads or other identifiers on the map thatthe route travels towards after the maneuver is performed at theselected juncture.

After analyzing the juncture relationships, the process synthesizes (at5740) instruction elements for the selected juncture using the analyzedjuncture data. The synthesized instruction elements are associated withthe selected juncture. In some embodiments, synthesized instructionelements include a set of text strings. As discussed above by referenceto FIG. 52, examples of such text strings include “at the end of theroad”, “turn right”, “onto 1^(st) St.”, “towards B St.”, and “for 0.1miles”.

In some embodiments, each text string is associated with an elementcategory. Examples of element categories include: “at” elements thatdefine the location at which the main maneuver will occur, “turn”elements that summarize the primary maneuver to make at the selectedjuncture, “onto” elements that identify the object (i.e., road) ontowhich the main maneuver will turn, “towards” elements that identify thenext object towards which the main maneuver will lead, and “for”elements that identify the distance between the selected juncture andthe next juncture. However, other embodiments might use additional,fewer, or different element categories than those listed here.

Next, the process determines (at 5750) whether any junctures remain tobe processed. When additional junctures remain for processing (i.e., forwhich elements have not yet been synthesized), the process 5700 returnsto operation S720 to select the next juncture. Once all of the junctureshave been processed, the process 5700 ends.

The output of the process 5700 is a set of instruction elements for eachjuncture of a route received from the mapping service server. In someembodiments, the mapping application on the client device then usesthese synthesized instruction elements to generate several textnavigation instruction variants for each juncture.

FIG. 58 conceptually illustrates a process 5800 of some embodiments forgenerating navigation instruction variants for display in differentcontexts. In some embodiments, the process 5800 is performed by amapping application that operates on a device (e.g., a mobile devicesuch as a smart phone or touchpad). In some embodiments, the mappingapplication performs the processes 5700 and 5800 in succession (i.e.,synthesizing the elements for each juncture and then using the elementsto generate instruction text).

As shown, the process 5800 begins by receiving (at 5810) sets ofinstruction elements that are each associated with a particularjuncture. As discussed above by reference to FIG. 57, in someembodiments the sets of instruction elements include text stringsassociated with element categories (i.e., “at”, “turn”, “onto”,“towards”, and “for” categories).

Next, the process 5800 selects (at 5820) an unprocessed set ofinstruction elements for a particular juncture of the route. While theprocesses 5700 and 5800 illustrate the mapping application firstsynthesizing the elements for each juncture and then subsequentlygenerating the instruction text, in some embodiments the mappingapplication performs all of the operations in a single loop over eachjuncture. That is, the mapping application of some embodiments selects ajuncture, synthesizes its elements, then generates the instruction textvariants before moving on to the next juncture.

For the selected set of instruction elements, the process 5800 combines(at 5830) the instruction elements of the set into instruction variants.As a first example of such a combination, the process 5800 may combine“at the end of the road” and “turn right” to produce a shorter textinstruction variant that reads, “At the end of the road, turn right.” Asa second example of such a combination, process 5800 may combine all ofthe previous text strings to produce a longer text instruction variantthat reads “At the end of the road, turn right onto 1^(st) St., towardsB St. for 0.1 Miles.” Some embodiments generate the same combinations ofelements for each juncture. For example, in some embodiments theapplication generates a first instruction text variant for each juncturefrom all of the elements, a second instruction text variant from onlythe “turn” and “onto” elements, a third instruction text variant fromthe “turn”, “onto”, and “towards” elements, etc. Other embodiments takeinto account other factors to generate different instruction textvariants from the synthesized elements.

After generating the instruction text variants for a selected juncture,the process 5800 ranks (at 5850) the instruction variants according tothe amount of information conveyed in each variant. In some embodiments,the application uses length (e.g., number of characters) as a proxy forinformation conveyed. Other embodiments prefer specific combinations ofelements over other combinations of elements, even if this methodologyresults in ranking a slightly shorter variant ahead of a longer variant.Some embodiments, for example, use specific combinations of elements foreach juncture, with each specific combination having the same rankingfor each juncture.

The process 5800 then determines (5860) whether any unprocessed sets ofinstruction elements remain (i.e., whether instructions have beengenerated for all of the junctures of the route). When additionalunprocessed sets of instruction elements remain, the process 5800returns to operation 5820 to select the next set of instructionelements. Otherwise, once all of the junctures have had theirinstructions generated, the process 5800 ends.

C. Navigation Instruction Software Architecture

As stated above, in some embodiments the maps, routes, and turn-by-turnnavigation are presented to a user by a navigation application thatoperates on a device (e.g., a handheld device such as a smart phone ortablet). The navigation application may be a stand-alone application insome embodiments, or integrated with the operating system of the device.FIG. 59 conceptually illustrates a system architecture that includes amapping and navigation application 5900 of some embodiments thatgenerates text instructions for different contexts. One of ordinaryskill in the art will recognize that the modules shown for theapplication 5900 are specific to the text instruction generationprocesses, and that the mapping and navigation application of someembodiments will include numerous additional modules (e.g., for mapdisplay, route display, additional aspects of navigation, etc.).

As shown, a mapping service server 5910 transmits route and juncturedata through a network 5915 to a network interface 5920 of the device onwhich the mapping and navigation application 5900 operates. In someembodiments, the mapping service server 5910 receives a route requestfrom devices on which the navigation application 5900 operates andgenerates route and juncture data for the requests.

The navigation application 5900 includes a juncture decoder 5930, aninstruction generator 5945, an instruction retriever 5960, a contextanalyzer 5965, a sign generator 5970, an arrow selector 5975, and adrawing engine 5980. The juncture decoder 5930 receives encoded junctureinformation 5925 for a route, decodes this information to arrive atinformation describing each juncture and the maneuvers performed at thejunctures, and synthesizes a set of instruction elements for eachjuncture in the route. In some embodiments, the juncture decoder 5930executes the process 5700 of FIG. 57, described above. The juncturedecoder 5930 decodes encoded junctures 5925 to generate decoded juncturedata 5935.

In some embodiments, encoded junctures 5925 and decoded junctures 5935may be stored on the device in random access memory or other volatilestorage, for use only during the navigation of the route, or in a morepermanent storage such as a hard disk or solid-state memory. As shown byexample table 5940, some embodiments store a set of text string elementsfor each juncture, e.g., in a table, array, or similar data structure.FIG. 52, described above, illustrates an example of such a table storedfor a particular set of junctures.

The instruction generator 5945 generates ranked text instructionvariants 5955 for display on a device based on the synthesizedinstruction elements 5935 received from the juncture decoder 5930. Insome embodiments, the instruction generator 5945 executes the process5800 of FIG. 58, described above. In some embodiments, the ranked textinstruction variants 5955 may be stored on the device in random accessmemory or other volatile storage, for use only during the navigation ofthe route, or in a more permanent storage such as a hard disk orsolid-state memory. FIG. 52, described above, also illustrates anexample of ranked instruction variants 5955 generated by the instructiongenerator 5945. The example table 5950 illustrates the results ofsynthesizing the elements 5940 into several text instruction variantsfor use in different contexts of the navigation displays.

The instruction retriever 5960 uses a context analyzer 5965 to determinewhich of the instruction variants to select for a particular display ofa maneuver, depending on the context in which the text instruction willbe displayed. These contexts may include different situations forrouting directions or different situations for turn-by-turn navigationinstructions (e.g., standard mode, lock-screen mode, when a differentapplication is open, when voice navigation is activated, etc.). In someembodiments, the context depends on several factors associated withclearly displaying the route maneuvers required for navigating theroute. For example, the context may be based on the amount of spaceavailable to display the text instruction (e.g., due to the size of thedevice on which the route directions are displayed), the conditionsunder which the indicator will be displayed (e.g., whether the maneuveris a current or future route maneuver, in which particular modality ofthe navigation application the sign will be displayed, etc.), or otherfactors. Many such contexts are shown above in subsection A of thisSection. The instruction retriever 5960 selects an instruction variantto use for a particular maneuver display and provides this informationto the sign generator 5970.

The arrow selector 5975 also uses the context analyzer 5965 to determinewhich of the directional indicators to use for a particular maneuver,depending on the context in which the indicator will be displayed. Thearrow selector chooses one of the graphical indicators described in theprevious section (e.g., either a complex or simple representation of amaneuver) and provides this selection to the sign generator 5970. Thesign generator 5970 generates a navigation instruction sign for displaythat includes the selected graphical indicator and instruction textvariant. The sign generator 5970 also uses the context analyzer resultsto generate other aspects of the sign, such as how often to update thedistance information and whether to use road sign shields in place ofroad names.

The drawing engine 5980 receives the generated signs from the signgenerator 5970 and incorporates the sign into a display to output to adisplay device (or to a frame buffer that feeds into the displaydevice). In some embodiments, the drawing engine 5980 incorporates thesign along with a 2D map, 3D map, or other GUI elements shown by themapping and navigation application.

V. Navigation Instructions when not in Navigation Application

A. Instructions when Device is Unlocked and Navigation is Operating inBackground

Some embodiments allow the navigation application to run in thebackground while other applications are running in the foreground. Theseembodiments provide unobtrusive navigation instructions in theforeground even while the main navigation application is running in thebackground and another application or an application launcher is runningin the foreground. Examples of applications running in the foregroundinclude voice-activated personal assistant, mail, browser, phone,calendar, or any other application available on the device.

The navigation application of some embodiments provides a navigation bar(sometimes called a “banner” or “navigation banner”) as well as aregular status bar on the screen. Some embodiments provide a navigationstatus bar when no navigation instructions are being provided andprovide a navigation instruction bar when navigation instructions arebeing given. FIG. 60 illustrates a user device display 6000 whennavigation is in operating in background in some embodiments of theinvention. The user device display 6000 is shown in four stages6001-6004.

In stage 6001, the display 6000 shows navigation application 6005, astatus bar 6080, and a button 6015. The status bar 6080 shows differentinformation such as battery level, time, reception bars, etc. In someembodiments, the status bar displays an indicator such as an arrow,which indicates that the navigation application or a map application isrunning. In this stage 6001, the navigation application 6005 is runningin the foreground until the device receives a selection (e.g., a click)on button 6015 that switches from the navigation application to theapplication launch view, which can itself be characterized as anapplication launching application. In some embodiments, there are othercontrols instead of or in addition to the button that switch thenavigation application to another application (e.g., the applicationlaunch view or other applications). The stage 6001 also shows that theroad names are displayed on road signs and not in banners. As mentionedabove, the mapping application of some embodiments may display the roadnames on the road and/or in the banners regardless of the mode in whichthe mapping application operates.

In stage 6002 application launcher 6075 is displayed in the foreground.The foreground application launcher 6075 has icons 6025 that have theirnormal functions (e.g., launching other applications) while thenavigation application runs in the background. In stage 6002 abackground navigation status bar 6010 is shown below the status bar6080. Some embodiments display the status bar 6080 and/or the navigationstatus bar 6010 in a different color (e.g., green) when navigation isrunning in background (as shown in stage 6002) than the status bar color(e.g., gray) when navigation is not running in background (as shown instage 6001). In other embodiments, the status bar 6080 is the same colorwhen the navigation application is running in the background, thenavigation application is off, or the navigation application is runningin the foreground. In some embodiments, the thickness of the navigationstatus bar is the same or approximately the same (e.g., 75% to 125% ofthe thickness) as the thickness of the status bar when the navigationapplication is not currently displaying a direction in a navigationinstruction bar.

The navigation status bar 6010 in some embodiments is both an indicatorthat the navigation application is running in the background and acontrol for bringing the navigation application to the foreground. Thenavigation status bar 6010 in some embodiments is not limited to beingdisplayed only with the application launching screen 6075, but rather isdisplayed below the status bar 6080 at the top of any application thatis running in the foreground.

In stage 6003, the navigation status bar 6010 is selected by touchingthe navigation status bar 6010 on the screen. Some embodiments alsoallow the selection of the navigation bar by other touch-based ormotion-based input devices as well as non-touch based or motion-basedinput devices. Some devices used for selection in some embodimentsinclude keyboards, mice, joysticks, touch-pads, and the like (e.g.,selection can be by a click from a mouse). The selection of thenavigation status bar 6010 (as shown in stage 6003) causes thenavigation application 6005 to return to the foreground in stage 6004.In addition to utilizing navigation status bar 6010 to return to thenavigation application (i.e., to bring the navigation application toforeground), in some embodiments the navigation bar has other functions.For instance, the navigation status bar 6010 is used in some embodimentsto provide navigation instructions (e.g., turn-by-turn directions) whilethe navigation application itself is still in the background. In otherembodiments, the navigation status bar is replaced at various times by anavigation instruction bar that provides instructions.

FIG. 61 conceptually illustrates a process 6100 of some embodiments forproviding directions while a navigation application is running in thebackground. FIG. 61 will be described with respect to FIG. 62, which isbriefly described first. FIG. 62 illustrates a user interface of someembodiments in which navigation instructions are given while thenavigation application is running in the background and anotherapplication is running in the foreground. The figure shows six stages6201-6206. The first stage includes status bar 6280, navigation statusbar 6210, and foreground application 6215. The remaining stages6202-6206 show the changes to the navigation status bar 6210 (e.g., itsreplacement by navigation instruction bars 6220-6250) as the device ismoved toward, and then passes a navigation point (sometimes referred toherein as a maneuver, some navigation points represent junctions in theroad).

As shown in FIG. 61, process 6100 displays (at 6105) the navigationapplication in the foreground. The process then determines (at 6110)whether the control (e.g., button 6015 of FIG. 60) has been activated.If not, the process keeps displaying the navigation application in theforeground until the control is activated (or in some embodiments, untilsome other control is activated or the device goes into a sleep mode).When the control is activated, the process displays (at 6115) theapplication launching mode in the foreground and displays (also at 6115)the navigation status bar 6210 to indicate that navigation is running inthe background. This is illustrated in stage 6201 in FIG. 62.

One of ordinary skill in the art will understand that in someembodiments, a navigation bar (a navigation instruction bar and/or anavigation status bar) appears at the top of some or all foregroundapplications, not just the application launching application. Theactivation of one or more controls in some embodiments causesapplications other than the launching application to move to theforeground. Furthermore, in some embodiments the navigation barcontinues to appear above foreground applications after switchingbetween one foreground application and another, and not just whenswitching directly from the navigation application to a particularforeground application. An example of a navigation bar being displayedabove another application is shown in FIG. 63, described below.

Process 6100 then determines (at 6120) whether the user device is near anavigation point (e.g., at a waypoint turn). While the applicationdetermines (at 6120) that the device is not near a navigation point thedisplay remains as shown in stage 6201 of FIG. 62.

Stage 6201 shows the state of a device when the navigation applicationis active as a background application and the foreground application6215 is the application launching screen. The navigation application hasnot been turned off, but instead has been left on in the background. Thevisible indication in stage 6201 of the navigation application being onin the background is the navigation status bar 6210. Also, someembodiments display the status bar 6280 in a different color from itsusual color when navigation is running in the background. In someembodiments the status bar 6280 and the navigation status bar 6210 areshown in various shades of green. In some embodiments, the colors orshades of one or both of the status bar and the navigation bar changeover time to draw attention to the fact that the navigation applicationis executing in the background.

At this stage 6201, the device (and the person or vehicle carrying thedevice) is far from the next navigation point. The application of someembodiments, including the one illustrated in FIG. 62 do not displayturning information for the entire span of time that the application isrunning in the background. In some such embodiments, when the device isnot near a navigation point (e.g., when no turn is imminent) the devicedisplays, in the navigation status bar 6210, “touch to view navigation”,or “touch to return to navigation” or some other message indicating thatselecting the navigation bar will bring the navigation application tothe foreground. In other embodiments, the navigation instructions aredisplayed whether or not the device is near a navigation point.

Referring back to FIG. 61, when process 6100 determines (at 6120) thatthe device is approaching the next navigation point, the process changes(at 6125) the navigation status bar 6210 to a navigation instruction bar6220 to display the new navigation instruction. This is shown in stage6202 of FIG. 62. In stage 6202, the device is approaching a navigationpoint (a left turn in 500 feet). In this stage 6202, the navigationinstruction bar 6220 displays navigation instructions, which include anarrow indicating a left turn and a distance (500 feet) to the left turn.Process 6100 then displays (at 6130) a countdown (in feet) until itdetermines (at 6135) that the navigation point has been passed.

In some embodiments, the navigation bars in stage 6201 and 6202 aretreated as separate entities that happen to occupy a similar place onthe screen. In such embodiments the navigation bar of stage 6201 can becharacterized as a “navigation status bar”, while the navigation barwith navigation instructions in stage 6202 can be characterized as a“navigation instruction bar” or a “navigation direction bar”. In someembodiments, the navigation instruction bar 6220 is thicker than thenavigation status bar 6210 (e.g., twice the thickness or more) andcovers up the status bar. In other embodiments, the navigation bar istreated as a single entity that expands (e.g., to twice its previousthickness or more) to cover or replace the status bar when thenavigation bar displays navigation directions.

In stages 6203 and 6204, as the device moves closer to the navigationpoint, the distance to the navigation point is counted down in thenavigation instructions in navigation instruction bars 6230 (100 feet)and 6240 (0 feet). In stage 6204, the instructions have begun to switchto the next instruction.

In stage 6204, the actual turn is taking place. The navigationinstructions in navigation instruction bar 6250 (shown in stage 6205)are replacing the previous navigation point instructions in navigationinstruction bar 6240 with the instructions for the next navigationpoint. In some embodiments, including the illustrated embodiment, thenavigation instructions are switched in a simulation of a flipping signwith multiple faces. Accordingly, instruction 6240 shows the instruction“0 feet turn left” as it begins to flip. In some embodiments, the signflips up, in some embodiments the sign flips down. In other embodiments,the device uses other transition methods to remove the old navigationinstruction in navigation instruction bar 6240 and replace it with thenew navigation instruction in navigation instruction bar 6250 (in stage6205). For instance, some embodiments simulate a new instruction slidingup, down, or sideways while the old instruction slides in the samedirection. Other embodiments simulate sliding the new instruction overthe old instruction. Still other embodiments simply have the oldinstruction disappear to be replaced by the new instruction.

When a navigation point is reached, process 6100 determines (at 6140)whether the final destination has been reached. If the final destinationhas been reached, the navigation ends (this is illustrated in FIG. 64,described below). If the final destination has not been reached thenthere is a new navigation point to display (at 6145). This is shown instage 6205 of FIG. 62.

Stage 6205 occurs just after the left turn has been completed. Thenavigation instruction in navigation instruction bar 6250 has fullyreplaced the navigation instruction in navigation instruction bar 6240.The new navigation instruction in navigation instruction bar 6250indicates a significant distance to the next navigation point. Asmentioned above, the applications of some devices are programmed todisplay navigation instructions primarily when the device is near anavigation point, not at all times. Accordingly, after displaying thenext navigation instruction in navigation instruction bar 6250 for apre-set period (or in some embodiments after a preset distancetraveled), the application in some embodiments returns to showingnavigation status bar 6210 in stage 6206 (and process 6100 returns tooperation 6115). However, when the new navigation point is determined(at 6150 of FIG. 61) to be near, the process 6100 immediately beginscounting (at 6130) down the distance to the next navigation point.Different applications of different embodiments use various differentdistances to determine whether to show the navigation status bar 6210 ornavigation instructions (e.g., instructions in navigation instructionbar 6220). In some embodiments, the applications switch on theinstructions at 1 mile, or half a mile, or a quarter mile, or 1000 feet,or 750 feet, or 500 feet, or 250 feet, or some other distance.

FIG. 63 illustrates a navigation bar displayed at the top of anapplication. The figure demonstrates that the navigation bar isdisplayed in applications other than the application launchingapplication. The figure is shown in stages 6301-6303. In stage 6301, thenavigation application is in the foreground and the user has entered acommand (e.g., double pushing a button 6310) to bring up a list ofapplications currently running in the background. In stage 6302, thedevice is displaying a set of icons 6320 representing applicationscurrently in the background. In some embodiments, the set of icons 6320pushes up the UI of the application in the foreground as shown. In otherembodiments, the UI of the application in the foreground gets overlaidwith the set of icons 6320 instead of being pushed up.

The second stage 6302 also shows that the user selects icon 6325commanding that the application represented by icon 6325 (e.g., a webbrowser) be moved to the foreground and the navigation application bemoved to the background. One of ordinary skill in the art willunderstand that this is just one of many ways that some embodimentsswitch the navigation application into the background and anotherapplication into the foreground. For example the user could switch tothe application launching view and launch an application, which wouldthen replace the application launching view as the foregroundapplication.

The web browser 6330 that the device switches to the foreground is shownin stage 6303. At the top of the screen is a navigation instruction bar6335 indicating that the navigation application is running in thebackground and directing the user to turn right in 50 feet. In someembodiments, the status bar and a navigation status bar (e.g., as shownin FIG. 60) will appear when the navigation application is not currentlyproviding a direction.

After following the navigation instructions shown by the device, theuser will reach his intended destination. FIG. 64 illustrates the userinterface of a device 6400 in some embodiments where the device reachesits destination while the navigation application is running in thebackground of another application. The figure shows three stages6401-6403. The first stage 6401 shows navigation instruction bar 6410,and foreground application 6440. As shown, the instructions innavigation instruction bar 6410 indicate “50 Feet Go Straight”.

Stage 6402 illustrates the user device 6400 when the destination isapproached. As shown in this stage, the navigation instruction bar 6410indicates “Destination on the Left”. Stage 6403 illustrates the userdevice 6400 after the destination is reached. As shown, navigationinstruction bar 6410 of stages 6401 and 6402 is removed from the screento indicate that the navigation instructions are finished and status bar6480 returns to the screen. In some embodiments, the navigationapplication remains on in the background, but not visibly displayed atthis stage 6403. In other embodiments the navigation applicationswitches itself off at this stage 6403. In still other embodiments, thedevice continues to display the navigation bar after the destination isreached. Furthermore, the navigation application of some embodimentsidentifies a location as the end of vehicular navigation and indicatesthat the rest of the journey must be completed on foot, which thenavigation application directs (e.g., in the navigation instructionbar).

The stage 6403 also shows that the icons 6490 have not moved. However,in other embodiments, the icons 6490 may move up to occupy at least aportion of the space that used to be occupied by the navigationinstruction bar 6410 at the previous stages in some embodiments, whenthe navigation instruction bar is removed from the screen.

As described above, the navigation status bar and the navigationinstruction bar are treated as distinct components in some embodiments.The above described figures show the navigation status bar below astatus bar. However, when the navigation application is running in thebackground, the status bar itself is replaced with a navigation bannerin some embodiments. This navigation banner is twice the height of theregular status bar in some embodiments. The navigation banner of someembodiments displays some or all of the same information as the statusbar it replaces. In some embodiments, the navigation banner displaysthat information when the device is not nearing a navigation point anddoes not display it when the device is nearing a navigation point. Whenthe device is nearing a navigation point, some or all of the statusinformation is removed so that a direction relevant to the upcomingnavigation point can be seen more clearly.

Devices that execute navigation applications of some embodiments includetelephonic devices. In some embodiments, when a telephone call is beingprocessed by the device, and the navigation application is running inthe background, data about the telephone call (e.g., call time) replacesa navigation status bar or the instruction bar with a telephone callstatus bar.

FIG. 65 illustrates interaction between a call status bar and anavigation instruction bar. The figure is shown in three stages6501-6503. In stage 6501, a call is going on while the device isdisplaying an application launching view. The call is indicated by acall status bar 6515 under the status bar 6510. The call status bar insome embodiments indicates that a call is ongoing, contains an indicatorof the duration of the call, and allows the user to select the callstatus bar to return to a screen view normally used for handling calls.In some embodiments, the original status bar 6510 (showing battery lifeetc.) turns to a color that indicates that a call is ongoing (e.g., redor green). In some embodiments, the telephone call status bar 6515 is asimilar color to a color that the original status bar displays duringthe call (e.g., both are shades of red or both are shades of green).

In some embodiments, the navigation instruction bar 6520 re-emerges andreplaces the phone data under some circumstances. In stage 6502, thedevice is near a navigation point. Therefore, the navigation instructionbar 6520 replaces the call status bar 6515 and the status bar 6510.After the navigation point is passed, the call status bar 6515 and thestatus bar 6510 are redisplayed as shown in stage 6503. In theillustrated embodiment of FIG. 65, the call status bar is redisplayed assoon as the navigation point is passed. However, in some embodiments,the phone call status bar is not redisplayed until after the nextnavigation instruction is displayed in the navigation instruction bar6520.

The stages 6402 and 6403 show that the icons 6490 have not moved.However, in other embodiments, the icons may move up or down to occupydifferent spaces depending on the presence of the call status bar 6515and the navigation instruction bar 6520.

B. Instructions when Device is Locked

1. Layout

In some embodiments, devices with multiple functions (e.g., mobilephones that run multiple applications) can be placed into locked modefrom various applications. In some embodiments, there are multiple waysto place a device into locked mode. The locked mode of some embodimentsis a mode with most of the controls disabled and with limitedfunctionality until the device is unlocked. This has the benefit in someembodiments of preventing the user from accidentally ending navigationmode prematurely. In some embodiments, unlocking the device requires aparticular gestural command on a specific part of the screen.

Some devices have a button that switches the screen off and/or puts thedevice into locked mode. Some devices have a timeout function thatswitches the screen off and/or puts the device into locked mode after acertain time has elapsed between user commands. Regardless of the waythat the applications get into locked mode, most such devices come outof locked mode with the same application running in the foreground asthe application running in the foreground when locked mode was entered.However, in the devices of some embodiments, regardless of whatapplication (or application launcher) is running in the foreground whenthe device is locked, if the navigation application is running in thebackground, then the application returns from locked mode directly intothe navigation application.

FIG. 66 illustrates a device 6600 of some embodiments that enters lockedmode with the navigation application running in the background and exitslocked mode with the navigation application running in the foreground.The figure shows the device 6600 in four stages 6601-6604. In stage6601, the application launcher 6620 in the foreground and the navigationapplication is running in the background. The navigation applicationrunning in the background is indicated by the navigation bar 6610 at thetop of the screen, just below the status bar 6615 and above theforeground application launcher 6620. As shown, in stage 6601 the userpushes a control 6690 to lock the screen.

In stage 6602, the device is in a locked mode (as indicated by theunlocking slider 6630 on the screen). In this stage, the map 6640 isshown on the locked screen and turn-by-turn directions are shown on theinformation bar 6650.

In stage 6603, the user has started to slide the unlocking slider 6630to the right in order to unlock the device. In this stage, the map 6640is displayed on the screen and turn-by-turn navigation directions areshown on the information bar 6650. In some embodiments (not shown), whenthe slider moves all the way to the right, the user is asked to enter apass code to unlock the screen. After the user successfully enters thepasscode, the screen is unlocked. In some embodiments, the directionsand/or the map are not shown under some circumstances in locked mode.For example, an interface for answering an incoming call may bedisplayed when a call comes in to the device and an interface fordealing with a call may be displayed when a call is in progress. Such aninterface may override the display of the directions in the informationbar, the display of the map, or both. Similarly, in some embodiments,other display views may replace the information bar, the map, or botheven though the navigation application is still running on the device.

However, after the screen is unlocked, the navigation map 6640 stays inthe foreground (instead of displaying application 6620 that was runningin the foreground prior to the locking of the screen). As shown in stage6604, the navigation application appears in full screen in theforeground. In this stage the screen is unlocked and navigationinstructions 6660 and the map 6640 are displayed on the screen. In someembodiments, the navigation application includes the map 6640 in thesame position as the map 6640 in the locked-screen view. Accordingly, insome embodiments, even for devices that ordinarily use a transition(e.g., a wipe or expansion of the new screen view from the center of thescreen) between locked-screen views and other views when returning fromlocked mode, the device in the transition from stage 6603 to stage 6604leaves the map in place and switches the other elements in the screen.That is, the map is constantly displayed during the transition fromstage 6603 to stage 6604, while the navigation bar 6610 and theunlocking slider 6630 disappear and the navigation instructions 6660appear instead. As stage 6604 shows, the device has returned from lockedmode directly into the navigation application, even though thenavigation application was running in the background, not the foregroundin stage 6601 before the device was locked.

FIG. 67 illustrates a device 6700 in some embodiments that enters lockedmode with the navigation application running in the foreground and exitsthe locked mode with the navigation application still running in theforeground. The figure shows the device in four stages 6701-6704. Instage 6701, the navigation application is running in the foreground anda map 6740 and navigation instructions 6760 are displayed on the screen.As shown, the user pushes a control 6790 to lock the screen.

In stage 6702, the device is placed into locked mode (as indicated bythe unlocking slider 6730 on the screen). In this stage, the map 6740 isshown on the locked screen and turn-by-turn directions are shown on theinformation bar 6750.

In stage 6703, the user has started to slide the unlocking slider 6730to the right in order to unlock the device. In this stage, the map 6740is displayed on the screen and turn-by-turn navigation directions areshown on the information bar 6750. When the slider moves all the way tothe right, the user is prompted (not shown) to enter the passcode tounlock the screen. After the user successfully enters the passcode, thescreen is unlocked. As mentioned above with respect to FIG. 66, in someembodiments, the directions and/or the map are not shown under somecircumstances in locked mode. For example, an interface for answering anincoming call is displayed in some embodiments when a call comes in tothe device and an interface for dealing with a call is displayed when acall is in progress. Such an interface overrides the display of thedirections in the information bar, the display of the map, or both.Similarly, in some embodiments, other display views may replace theinformation bar, the map, or both even though the navigation applicationis still running on the device.

As shown in stage 6704, the navigation application appears in theforeground. In this stage the screen is unlocked and the map 6740 andthe navigation instructions 6760 are displayed on the screen. In someembodiments, the navigation application includes the same map 6740 inthe same position as in the lock-screen view. Accordingly, in someembodiments, even for devices that would have a transition screen (e.g.,a wipe or expansion from the center) when returning from locked mode,the device in the transition from stage 6703 to stage 6704 leaves themap in place and, in some embodiments, switches the other elements inthe screen. That is, the map is constantly displayed during thetransition from stage 6703 to stage 6704, while the information bar 6750and the unlocking slider 6730 disappear and the navigation instructions6760 appear on the display. As stage 6704 shows, the device has returnedfrom locked mode back into the navigation application.

In the preceding two figures, the user pushes a control to enter alocked mode. In some embodiments, the user pushes such a control to turnthe display off. Later, when the display is turned back on, either bypressing the same control again, or by pressing another control, thenthe device shows the locked mode when the display turns on again.Similarly, in some embodiments, the device has a timeout function thatturns the display off after some particular amount of time has passedwithout the device receiving a command. In some embodiments, the deviceis in locked mode when the display turns on after such a lockout.

In addition to (or in some embodiments instead of) giving navigationinstructions on a navigation bar when other applications are in theforeground, the navigation applications of some embodiments also providenavigation instructions while the device is in a locked mode. FIG. 68illustrates a navigation application giving directions on a lockeddevice in some embodiments of the invention. The figure is shown in fourstages 6801-6804. In stage 6801, the device screen is displaying statusbar 6880, navigation bar 6810, map 6812, location indicator 6814, andunlocking slider 6816. One of ordinary skill in the art will understandthat other configurations and controls are possible within the scope ofsome embodiments.

In stage 6801, the device is close to the next navigation point,therefore navigation bar 6810 displays instructions to turn right in 500feet. In some embodiments (including the illustrated embodiment) thenavigation bar 6810 is translucent, allowing feature of the map 6812 tobe seen through the navigation bar 6810. The location indicator 6814indicates the location of the device, relative to the features of map6812. The map itself includes the road the device is on (Curb Road), andthe road that the navigation application is directing the user towards(T Road). Also displayed is a dark line 6818 showing the directed travelof the device and a lighter line 6819 showing the previous locations ofthe device along the navigation application's selected route. Theunlocking slider 6816, unlocks the device when activated. The unlockingslider 6816 is, however, unused in this figure.

As the device reaches a point 250 feet from the navigation point, thenavigation bar changes instructions as displayed in navigation bar 6820in stage 6802. The location indicator 6814 is at the same location, butthe map 6812 has moved down relative to the location indicator 6814. Thenew location of the map relative to the location indicator 6814 isanother way that the navigation application shows that the device hasmoved closer to the navigation point.

Similarly, in stage 6803, the navigation bar 6830 indicates that thenavigation point is only 100 feet away and the location indicator 6814is closer to the turn on the map. Finally, in stage 6804, the device hasgone around the corner and navigation bar 6840 is displaying the nextnavigation instruction. Although the transition between navigationinstructions is not shown in this figure, in some embodiments thetransition is similar to the described transition in background mode(with one instruction seemingly flipping up as if on a one side of asign and being replaced by another that seems to be on another side ofthe sign). In other embodiments, other transition methods are used toremove the old navigation instruction 6830 and replace it with the newnavigation instruction 6840 (in stage 6804). For instance, someembodiments simulate a new instruction sliding up or sideways while theold instruction slides in the same direction. Other embodiments simulatesliding the new instruction over the old instruction. Still otherembodiments simply have the old instruction disappear and be replaced bythe new instruction.

The new instructions are not the only indication that the turn has beenmade. The map 6812 has rotated so that the direction that the device istraveling in (along T Road) is shown on the map 6812 as being up. Thelighter line 6819 on the map 6812 now includes the corner that thedevice has just turned.

Although the location indicator 6814 is shown in FIG. 68 as always beingthe same size, in some embodiments, in one or both of the locked modeand the regular navigation mode, the location indicator is a differentsize depending on the zoom level. For example, in some embodiments, themore the map is zoomed in the larger the location indicator becomes.Similarly, the location indicator 6814 is always shown as having anarrow. However, in some embodiments, the arrow is not shown under somecircumstances. For example, in some embodiments, when the device is in abuilding (or otherwise off all the roads) rather than on a road, thearrow is not shown. The location indicator 6814 is shown as opaque inFIG. 68, however, in some embodiments, the location indicator istranslucent, semi-transparent, or transparent so as to show roads“underneath” it.

While operating in locked mode, the navigation application of someembodiments provides directions until the device reaches itsdestination. FIG. 69 illustrates the locked mode view of someembodiments when the device reaches its destination. The figure is shownin four stages 6901-6904. In stage 6901, the map 6940 shows lighter line6919 behind the current location indicator 6914. In front of locationindicator 6914, darker line 6918 ends at a circle 6912 that indicatesthe destination. According to navigation bar 6910, the destination is 50feet ahead.

Once the device reaches its destination in stage 6902, the navigationbar 6920 shows that the destination is on the right, darker line 6918 isno longer shown on the map 6940. In some embodiments, the device thendisplays a message that the device has “arrived” as shown in stage 6903.The navigation application then, in stage 6904, releases the lockedscreen to whatever its default configuration is when the navigationapplication is not providing navigation instructions. In the illustratedembodiment, the default configuration includes a time and date indicator6930.

This figure illustrates the locked mode view in a 2D map. However, themapping application of some embodiments may operate in the locked modewhile showing the map in 3D.

2. Notification Management

In some embodiments, devices notify their users of incoming textmessages and other noteworthy events. Even when such devices are in alocked mode, some such devices can still display notifications. However,leaving a notification on the screen for an extended period of time maydistract from navigation instructions also being displayed on thescreen. Accordingly, some embodiments briefly display a notification onthe screen and then make the notification accessible, but not visible.In some embodiments, there is a visible but unobtrusive sign indicatingthat there is a notification item waiting to be read. FIG. 70illustrates a locked view notification system of some embodiments. Thesystem is shown in four stages 7001-7004.

In stage 7001, the navigation bar 7010 is below the status bar 7080 atthe top of the screen displaying a navigation instruction. Anotification message 7012 is displayed on the screen over the map 7040to indicate that a text message has been received. The actual textmessage is not displayed in the illustrated embodiment, but embodimentsthat display the actual text message are within the scope of theinvention. Some embodiments display a name (if known) of the textmessage sender or a phone number from which the text message originatedin notification message 7012.

The application of some embodiments displays the notification for apreset length of time before the notification disappears, leaving thefull map 7040 visible again. Some applications display the notificationfor less than 5 seconds, some for 5 seconds, and some for more than 5seconds. Once the notification disappears, a drawer control 7022 appearsin stage 7002 in the navigation bar 7010. The application of someembodiments, including the illustrated application, allows the drawercontrol 7022 to be expanded (e.g., by a touch gesture that drags down onthe drawer control) in order to open a list of received notificationitems. Applications of other embodiments allow the drawer control to betapped to open the list, or double tapped to open the list. Similarly,other applications allow the drawer control to be selected by othermeans, (e.g., a selection such as a click on an associated cursorcontrol device).

In the illustrated embodiment, the drawer 7034 is shown as open in stage7003. In this stage 7003 the drawer, in this case including only onetext message 7032 and one missed call 7033, is shown in a list whichcovers the map from the bottom of the navigation bar 7010 to the top ofthe unlocking slider 7015. However, in some embodiments the drawer istranslucent, semi-transparent, or transparent, allowing the map to beseen through the list. In some embodiments, the drawer only partiallycovers the map 7040 (e.g., covers half the map, or only that portion ofthe map needed to show all the text messages and other notificationitems in the drawer). In some embodiments, if a new message ornotification that would normally be sent to the drawer arrives while thedrawer is open, the message will be added to the drawer right away (withor without displaying a pop up notification in various embodiments).

When the list of messages is too long to fit on the screen the list canbe scrolled up and down if necessary in some embodiments. When the useris finished looking at the list of messages, the user can close thedrawer by activating a control (e.g., a hardware or on screen controlsuch as a control that turns off the display) in some embodiments. Insome embodiments, the drawer will remain open until the user turns thedisplay off, and then back on again. The control could also include anynumber of controls activated by a gestural command such as a tap on thelist or elsewhere on the screen, a double-tap, or a sliding gesture(e.g., a sliding gesture up with part or all of the drawer as thecontrol) in some embodiments. The control could also include a button orother component of a mouse or other cursor control device, etc., in someembodiments.

Also, in addition to or instead of having a control to close the drawer,some embodiments display the opened drawer for varying lengths of timebefore it disappears, leaving the full map 7040 visible again as shownin stage 7004. Stage 7004 includes drawer control 7022. However, in someembodiments, after the drawer 7034 is closed, the drawer control 7022 isnot shown until a new message arrives.

After the drawer is closed, if and when another text message ornotification arrives, the stages 7001-7004 repeat with that new message,assuming that the navigation is still active. In some embodiments stage7004 happens only if the user closes the drawer. If the drawer remainsopen, then the display remains in stage 7003 in some embodiments.Furthermore, the drawer open stage 7003 may not immediately followstages 7001 and 7002. In some embodiments, if the user does not open thedrawer, stages 7001-7002 are repeated with each of multiple messagescoming in and the drawer remaining closed with the drawer control 7022displayed as the new message notifications appear.

In some cases, a user may decide to unlock the device before opening thedrawer 7034. In some embodiments, the normal behavior of the device whencoming out of locked mode with notifications is to list thenotifications on the screen. However, in some embodiments, when thenavigation application is running, opening into the navigationapplication takes priority over displaying the notification messages.Therefore the device of those embodiments unlocks and opens into thenavigation application rather than opening into a list of notificationmessages. In some such embodiments, a user can choose to open the listof notification messages after the navigation application is opened.FIG. 71 illustrates the viewing of notification messages after unlockinga device in some embodiments of the invention. The figure is shown insix stages 7101-7106.

In stage 7101, the navigation bar 7110 is below the status bar 7180 atthe top of the screen displaying a navigation instruction. Anotification message 7112 is displayed on the screen over the map 7140to indicate that a text message has been received. The actual textmessage is not displayed in the illustrated embodiment, but embodimentsthat display the actual text message are within the scope of theinvention. Some embodiments display the name of the sender, the phonenumber of the sender, or both in notification message 7112. Theapplication of different embodiments displays the notification forvarying lengths of time before it disappears, leaving the full map 7140visible again. Some applications display the notification for less than5 seconds, some for 5 seconds, and some for more than 5 seconds.

Once the notification disappears, a drawer control 7122 appears in stage7102 in the navigation bar 7110. Stage 7101 is identical to stage 7001of FIG. 70. However, in stage 7102, rather than opening the drawer 7122,the user instead unlocks the device with the unlocking slider 7116. Theuser has unlocked the device with the navigation application running inthe background, therefore, the navigation application appears in theforeground in stage 7103. As shown, the navigation application takespriority over displaying the notification messages.

The navigation application in some embodiments does not show a drawercontrol. However, by dragging the top center of the screen down (asshown in stage 7104) the user can cause the drawer 7144 to come down (asshown in stage 7105). In some embodiments, the drawer control 7122appears under the dragging finger as the finger drags the drawer 7144down. In other embodiments, when the navigation application is in theforeground, multiple drags must be employed. For example, one draggesture at the top of the screen is used to expose the drawer control7122 and a separate drag gesture on the drawer control 7122 is used toopen the drawer in some embodiments. Stage 7105 shows the drawer 7144fully extended and covering the entire screen. Text message 7152 appearsat the top of the screen.

In some embodiments, the drawer stays open until the user either closesthe drawer (at which point the navigation application appears again) orlocks the device. In some embodiments, the drawer can be closed bypulling up the drawer control 7122. In other embodiments, the drawercannot be closed by pulling up the drawer control up 7122, but can beclosed by some other control (e.g., a button or a gestural command). Forexample the device can be locked, e.g., by activating a control 7190which also closes the drawer in some embodiments. Some embodiments alsoautomatically close the drawer after a pre-determined amount of time. Insome embodiments, after the drawer is opened, either in locked mode orunlocked mode, once the drawer is closed the drawer is emptied and is nolonger accessible from the locked mode view, as shown in stage 7106, inwhich the drawer control 7122 is no longer present. That is, the drawercontrol 7122 will only be displayed again when a new notification isreceived. However, in other embodiments, the drawer control 7122 is notremoved, is only removed when certain methods of closing it areemployed, or is removed if the drawer is opened in the unlocked mode,but not if the drawer is opened in the locked mode.

In some embodiments, the drawer displays messages of different types inseparate areas. For example, some embodiments display text messages in aseparate area from “missed call” messages. In some embodiments thedrawer displays different types of messages in separate areas when it isopened in the unlocked mode, but the drawer in the locked mode does notdisplay different types of messages in separate areas. In otherembodiments the drawer displays different types of messages in separateareas when it is opened in the unlocked mode and the drawer in thelocked mode also displays different types of messages in separate areas.In other embodiments the drawer in locked mode uses separate areas fordifferent message types and the drawer in unlocked mode does not. Inother embodiments neither drawer separates message types.

3. Dynamically Turn on

Power saving is a feature of some embodiments of the application. Insome embodiments, the navigation application operating in locked modeswitches the screen on only when the device is approaching a navigationpoint or receives a notification. FIG. 72 illustrates a process 7200 forswitching the device screen on when approaching a navigation point insome embodiments of the invention. FIG. 72 will be described withrespect to FIG. 73, which will be briefly described first. FIG. 73illustrates multiple stages that a device goes through when no commandsare given to it while a navigation application runs in the background insome embodiments of the invention. FIG. 73 is illustrated in six stagesfrom 7301-7306. The various stages will be described at the appropriateplaces during the description of FIG. 72.

Process 7200 of FIG. 72 begins before the screen shuts off by displaying(at 7205) an application with the navigation application running in thebackground. Stage 7301 of FIG. 73 illustrates the pre-locked state ofthe device. This stage 7301 includes a foreground application 7312 (theapplication launching view) with the navigation bar 7310 below thestatus bar 7380 at the top of the screen, indicating that the navigationapplication is running in the background.

In some embodiments a device turns the screen off and enters locked modewhen it has received no commands in a pre-specified amount of time(e.g., 5 minutes, 15 minutes, etc.). The process determines (at 7210)whether any controls have been activated in the amount of timepre-specified for locking the device and turning of the screen. If anycontrols have been activated (other than one that shuts down the displayand/or locks the device right away) then the device resets its countdownto going into display off and locked mode.

When the process determines that enough time has passed, the processturns off (at 7215) the screen. In some embodiments, instead of or inaddition to the timeout screen deactivation, there is a control that theuser can select (e.g., a button) that puts the device into locked mode.In some embodiments, the timeout screen deactivation occurs when someapplications are running, but not when other applications are running.For example, in some embodiments, when the navigation application isrunning in the foreground the device does not shut down the screen aftera preset time. Furthermore, in some embodiments, the device doesn'ttimeout when the navigation application is running in the backgroundeither.

Operation 7215 is illustrated in stage 7302 of FIG. 73. Stage 7302 showsthe screen in black because it has been turned off either by a timeout,a control, or in some other way. While the screen is off and the devicetravels toward the next navigation point the process 7200 repeatedlydetermines (at 7220) whether the device is near the next navigationpoint. If the device is not near the next navigation point, the devicewill keep checking whether it is near the navigation point. “Near” meansdifferent distances in the application of different embodiments.

In different embodiments, the device determines that it is near anavigation point when the device is 1000 feet from the navigation point,or 500 feet, or 250 feet, or any other particular distance. Once process7200 determines (at 7220) that the device is near the navigation point,the process turns on (at 7225) an ambient light sensor. In someembodiments the ambient light sensor is part of a camera of the device.In other embodiments, the ambient light sensor is not part of a cameraof the device. In some embodiments, the ambient light sensor is on atall times. In some embodiments, the ambient light sensor is a passiveelement that doesn't need to be powered on to function. The ambientlight sensor determines how much light is present around the device. Ifthere is a large amount of light, then the screen would have to beturned on at a high level of brightness to be seen against the existinglight. However, if there if a low amount of ambient light, then thescreen could be turned on at a dimmer level and still be bright enoughto be seen in the lower ambient light.

Once the light level is determined, the process 7200 turns on (at 7230)the screen at a brightness level in accord with the ambient light levelsdetected by the ambient light sensor. The screen then displays (at 7235)a countdown to the next navigation point. This is illustrated in stage7303 of FIG. 73. The figure shows navigation bar 7330 with an arrowindicating a right turn and instructions to turn right in 1000 feet. Theprocess then determines (at 7240) whether the navigation point has beenpassed. If the navigation point has not been passed, then the process7200 returns to operation 7235. The process then continues to displaythe countdown to the next navigation point. Part of the countdown isshown in stage 7304 in FIG. 73. In stage 7304, navigation bar 7340indicates that there are 200 feet left to the right turn. Once thedevice passes the navigation point (in this case making the right turn),process 7200 determines (at 7245) whether the device is at itsdestination. If the device is at its destination, then the navigationprocess ends. If the device is not at its destination then the processdisplays (at 7250) the next navigation instruction. This is illustratedin stage 7305 in FIG. 73. In this stage navigation bar 7350 displays 2.8miles, go straight.

If the process 7200 determines (at 7255) that the next navigation pointis near, then the process returns to operation 7235 and counts down tothe next navigation point. However, that is not the case in FIG. 73. Ifthe process determines (at 7255) that the device is not near the nextnavigation point, then the process 7200 turns the screen off (at 7215).This is shown in stage 7306 which shows the dark screen. One of ordinaryskill in the art will understand that the words “Power saving mode” inFIG. 73, stages 7302 and 7306 are meant to conceptually illustrate thatthe display is turned off and that they are not physically displayed onthe screen during power saving mode in some embodiments.

The above described figure shows the device switching the display on asit nears predetermined navigation points, and switching the display offwhen it is not nearing the preset navigation points. However, in someembodiments, the device also turns the display on if the user deviatesfrom the prescribed route (e.g., the user takes a wrong turn). In somesuch embodiments, the device displays a “rerouting” message until thedevice has calculated a new route. In some embodiments, the device thendisplays the next navigation instruction and then turns off the displayunless the next navigation point is within the threshold distance.

In a similar manner to the way the navigation application of someembodiments turns on the screen in locked mode when the deviceapproaches a navigation point, the device of some embodiments turns onthe screen when a notification is received while the navigation programis running. FIG. 74 conceptually illustrates a process 7400 of someembodiments for turning on the screen when a notification message isreceived. Process 7400 will be described with reference to previouslydescribed FIG. 70. Process 7400 begins by turning the screen off (at7405). The screen can be turned off for any of the reasons discussedwith respect to FIG. 72. The process then waits (at 7410) until itreceives a notification. When the process 7400 receives a notification,it turns on (at 7415) the ambient light sensor (as described above inoperation 7225 of FIG. 72). The process then turns on (at 7420) thescreen at a brightness set according to the ambient light level asdetected by the ambient light sensor. The process then displays (at7425) the notification. This is shown in FIG. 70 in stage 7001 as popupmessage 7012. The process then puts (at 7430) the notification in adrawer as described with respect to stage 7002 of FIG. 70.

The process then determines (at 7435) whether the drawer has been opened(e.g., by the user sliding a drawer control 7022) before a time limit.If the drawer has not been opened within the time limit then the processturns the screen off (at 7405) again. If the drawer has been openedbefore the time limit, then the messages are displayed (at 7440), e.g.,as shown in FIG. 70 (as stage 7003, with message 7032 displayed). Theprocess then determines (at 7445) whether the drawer has been closed. Ifthe drawer has been closed then the process returns to operation 7405and turns off the screen after a timeout period. That is, in theapplications of some embodiments, the application waits for some amountof time after the drawer is closed before turning off the screen.

In some embodiments, if the process 7400 determines (at 7445) that thedrawer remains open, then the process determines (at 7450) whether atimeout period has been reached. If the timeout period has not beenreached, then the process continues to display (at 7440) the messages.If the time limit runs out before the drawer is closed by the user, thenthe process turns the screen off (at 7405). In some embodiments, if theuser is sending commands to the device (e.g., scrolling through themessages) then the countdown to the time limit will not begin until thedevice stops receiving commands from the user.

One of ordinary skill in the art will understand that although theflowcharts for process 7400 of FIG. 74 and the process 7200 of FIG. 72are being described separately, in some embodiments, they proceedsimultaneously and the screen will be on when either one of theprocesses requires it. In some cases it will already be on fornotification reasons when a navigation point becomes near. In thesecases, rather than switching on (at 7230) as process 7200 dictates, thescreen would simply remain on, even if process 7400 required that itturn off (at 7405) Similarly, processes 7200 and 7400 will continue insome embodiments until either the device is unlocked, or the destinationis reached (as shown in operation 7245 of process 7200 in FIG. 72).

As described above, the device in locked mode has a limited number ofactive controls. However, in some embodiments, while the locked mode isoperative, the map on the lock screen can be moved to one side or up anddown, to a greater or lesser degree, by a gestural command in thedirection that the user wishes to move the map. In some embodiments,when the device is released, the map returns to its default position.

VI. Hands-Free Navigation

A. Voice Guidances

In addition to presenting information in visual form and receivinginputs and commands through various touch-based or motion-based inputdevices (e.g., keyboard, mouse, joystick, touch-pad, touch-sensitivescreen, and the like), the navigation application of some embodimentssupports alternative modes of user interactions that do not require auser's visual attention and/or physical movements of the user's body orhands. For instance, in some embodiments, the navigation applicationincludes an interactive map application (or interactive navigationapplication) that provides information to the user in an audible form(e.g., as natural language speech), and receives user inputs andcommands in a verbal form (e.g., as natural language speech). By freeingthe user's visual attention and physical movements from the interactivemap, the interactive map allows the user to engage in other activities(e.g., driving, walking, surveying the surrounding environment, orpacking for a trip) while obtaining information from the interactivemap. In addition, through an audio/verbal user interface of theinteractive map, the user is able to invoke and solicit information orassistance from the interactive map more readily (e.g., as soon as theneed for the information or assistance arises) without disengaging fromthe user's current activities.

1. Accessing Interactive Map and Navigating when Lock-Screen is Active

In some embodiments, a mobile device implements a lock-screen thatprevents access to various applications installed on the mobile deviceuntil a password or other input is received from the user. The mobiledevice optionally allows the user to verbally invoke and access theinteractive map installed on the mobile device without unlocking thelock-screen. In some embodiments, a voice-activated service is initiatedby activating a button or control. In other embodiments, when the voicelevel received at the device audio input is louder than a certainthreshold (and/or natural language words are recognized by the device)the voice-activated service is automatically activated.

In some embodiments, while the device lock-screen is active, the devicereceives a speech input requesting access to the interactive map. Thespeech input may be a verbal request for directions, a verbal request toperform a local search (e.g., a search for restaurants, gas stations,lodging, etc.), or simply a verbal command to activate the interactivemap. In response to the speech input, the device makes at least a subsetof functionalities (e.g., providing directions and performing searches)of the interactive map available to the user through anaudio-output-and-speech-input user interface without deactivating thedevice lock-screen. In some embodiments, the interactive map provides anaudio-only output through the lock-screen in response to the user'sspeech input.

In some embodiments, in an audio-only mode of operation, touch-based andkeyboard-based interactions with the interactive map are disabled. Byallowing the user to access the interactive map directly from thelock-screen through an audio-only user interface, the interactive map ismade more accessible to the user without significantly compromising thesecurity of the user device. In some embodiments, in response to thespeech input, visual information (e.g., a list of search results) isprovided to the user on the lock-screen along with an audio output(e.g., a reading of the information to the user). In some embodiments,the device processes the user's speech input to determine the identityof the user and whether access to the interactive map should be allowed.

In some embodiments when voice-activated navigation is not used,navigation requires at least three steps: finding several results(search), showing directions to the results or showing several routes toa single destination address (showing overview), and then startingnavigation using a selected route (showing turn-by-turn directions).However, with voice-activated navigation, some embodiments anticipatehands-free interaction and initiate navigation with a single searchresult. In order to facilitate voice-activated (or hands-free)navigation, these embodiments only display one route (instead of theusual several routes).

For instance, interactive navigation finds a short route using freeways,a longer route using alternative freeways, and a route that does not usefreeways to get from the current location to a particular destination.Some embodiments select one of several routes found (e.g., based on adefault set up, user preferences set ups, past user preferences, etc.)during voice-activated navigation and optionally display an overview ofthe route and wait for the route to be loaded. Anticipating a hands-freeinteraction, the single route is displayed, and the display transitionsinto full-screen turn-by-turn navigation display. As described below,when several destinations (e.g., several gas stations along the route)are found during a search, the voice-activated service in someembodiments uses a list reading mechanism to cycle through the resultsin a sequential fashion.

FIG. 75 conceptually illustrates a process 7500 for performingvoice-activated interactions with the interactive map in someembodiments of the invention. As shown, the process receives (at 7505) avoice command to start interactions with the interactive map. The voicecommand may be commands such as “go to Dodgers Stadium” to go to adestination, “find Mediterranean restaurants” to start a search, or“start navigation” to start the interactive map.

The process then sets (at 7510) navigation mode to voice-activated mode.The process then determines (at 7515) whether the device lock-screen isactive. If yes, the process proceeds to 7540, which is described below.Otherwise, the process loads (at 7520) an overview of the route. In someembodiments, the process only displays one route (instead of the usualseveral routes) in order to facilitate voice-activated (or hands-free)navigation. In some embodiments when voice-activated navigation is notused, navigation requires at least three steps: finding several results(search), showing directions to the results or showing several routes toa single destination address (showing overview), and then startingnavigation using a selected route (showing turn-by-turn directions).However, with voice-activated navigation, some embodiments anticipatehands-free interaction and initiate navigation with a single searchresult. As described further below by reference to FIGS. 84 and 86, someembodiments cycle through several search results in batch fashion inorder to allow the user to select a search result. In these embodiments,after the user selects a particular destination, a single route fromseveral possible routes is set to the selected destination. Theseembodiments display only one route even though several routes are foundand all of the found routes would have been displayed if voice-activatednavigation were not in use.

The process then transitions (at 7525) from overview screen to fullscreen turn-by-turn display. The process then updates (at 7530) thescreen and provides turn-by-turn audio and visual directions. Theprocess then determines (at 7535) whether navigation has ended (e.g.,the destination is reached). If yes, the process exits. Otherwise, theprocess proceeds to 7530, which was described above.

FIG. 76 illustrates a user device 7600 when lock-screen is not active insome embodiments of the invention. The figure shows four stages7601-7604. As shown in stage 7601, the user activates voice-activatednavigation (e.g., by touching the button 7605) and makes a verbalrequest (as shown by arrow 7610) to navigate to office. In stage 7602,the screen displays the audible interaction between the user and thevoice-activated service. In some embodiments, the voice-activatedservice prompt (in this case, “What can I help you with?”) and theuser's verbal request are converted to text and are displayed on thescreen in order to show to the user how the verbal command isinterpreted by the voice-activated service. Display of the audiointeractions between the user and the voice-activated servicefacilitates communication, e.g., in noisy places. In order to simplifythe figures, the screen that shows the transcript of the communication(such as the screen in stage 7602) is not shown for every communicationbetween the user and the voice-activated service in some of the figuresdescribed in this Specification.

Also, some embodiments do not display the transcript of thecommunication between the user and voice-activated service on the screenduring navigation so that the display of the map or navigationdirections is not disrupted. Yet other embodiments display thetranscript of the communication on the same screen (e.g., whennavigation screen is displayed) instead of using a separate screen suchas the screen shown in stage 7602.

The receipt of the verbal navigation command results in an overviewscreen 7615 being displayed, as shown in stage 7603. Some embodimentsdisplay only one route 7635 when navigation is activated by a verbalcommand to facilitate hands-free navigation. As shown, the route isidentified by two markers or pins. One marker 7640 identifies the startof the route and the other marker 7645 identifies the end of the route.After a short delay, the display transitions to a full screenturn-by-turn display 7620 as shown in stage 7604. The voice-activatednavigation service continues to provide audible directions (as shown byarrow 7625) as well as visual turn-by-turn directions (as shown by arrow7630).

Referring back to FIG. 75, when the process determines that lock-screenis active, the process determines (at 7540) whether an option is set tonavigate while the screen is locked. Some embodiments provide auser-selectable option for allowing navigation while the screen islocked. Other embodiments always allow at least limited navigationfunctionalities when lock-screen is active. When the option does notallow navigation while lock-screen is active, the process exits.

Otherwise, the process determines (at 7545) whether the audio command tostart navigation is recognized as an authorized user. Some embodimentsuse voice recognition to compare the voice received (at 7505) for theaudio command with voice samples from authorized users of the device inorder to prevent an unauthorized user who has gained access to thedevice with locked screen to use the device. The embodiments that do notcheck for authorized users' voice bypass operation 7545. If the voice isnot recognized, the process ends. Otherwise, the process selects oneroute to the destination. As described above, some embodiments onlydisplay one route (instead of the usual several routes) to a particulardestination in order to facilitate voice-activated (or hands-free)navigation. In these embodiments only one route is displayed even thoughseveral routes are found and all of the found routes would have beendisplayed if voice-activated navigation were not in use. When there ismore than one destination (e.g., several Italian restaurants are foundalong the route), some embodiments (as described further below byreference to FIGS. 84 and 86) cycle through several search results inbatch fashion to allow the user to select one of the search results.After the user selects a particular destination, a single route fromseveral possible routes is set to the selected destination.

The process then determines (at 7550) whether navigation through thelock-screen is allowed only by audio. If yes, the process proceeds to7570, which is described below. Otherwise, the process shows (at 7555)navigation running in lock-screen using at least a subset of navigationfunctionalities such as providing directions and showing a list ofsearch results. The process also provides (at 7560) audible informationsuch as turn-by-turn directions, reading of search information to theuser, etc.

FIG. 77 illustrates a user device 7700 with lock-screen active in someembodiments of the invention. The figure is shown in four stages7701-7704. In stage 7701 the screen is locked. Lock-screen requires (asshown by the unlocking slider 7705) unlocking the screen and entering apasscode to unlock the screen in order to access different applications.However, the user device allows the user to verbally invoke and accessthe interactive map installed on the mobile device without unlocking thelock-screen.

As shown in stage 7702, the user activates voice-activated navigation(e.g., by touching the button 7715) and makes a verbal request (as shownby arrow 7710) to navigate to home. In stage 7703, the interactionsbetween the user and the voice-activated service are transcribed on thescreen. In stage 7704, the voice-activated service utilizes theinteractive map application to display the map 7720 with a single route7780 displayed and to start providing turn-by-turn navigationdirections. Visual directions (as shown in information banner 7770) aswell as audible instructions (as shown by arrow 7725) are provided insome embodiments. Some embodiments display an overview of the route(similar to screen 7615 described above) and after a short delaytransition to a screen to show turn-by-turn directions in lock-screen.Other embodiments do not show the route overview when lock-screen isactive and directly transition to the turn-by-turn navigation screen.Also, since the user request (i.e., go home) results in one destinationonly, the route is displayed without any further interactions with theuser. On the other hand, when there is more than one destination found(e.g., in response to a request to find a hotel) the user is allowed toselect one of the search results as described below by reference toFIGS. 84, 85A-85D, and 86.

Referring back to FIG. 75, the process then determines (at 7565) whetherthe navigation has ended (e.g., when a destination is reached or thenavigation is stopped by the user). If so, the process ends. Otherwise,the process proceeds to 7555, which was described above.

When the process is allowed in audio-only, the process provides (at7570) audible information such as turn-by-turn directions, reading ofsearch information to the user, etc. The process then determines (at7575) whether the navigation has ended (e.g., when a destination isreached or the navigation is stopped by the user). If so, the processends. Otherwise, the process proceeds to 7570, which was describedabove.

In some embodiments, when lock-screen is active and navigation isallowed only through audio, all other user inputs such as throughtouch-based or motion-based input devices are not allowed. FIG. 78conceptually illustrates a process 7800 for providing voice-activatednavigation while lock-screen is activated in some embodiments of theinvention. This process is utilized in some embodiments in conjunctionwith other processes used by voice-activated service in order todetermine whether only voice-activated commands should be allowed. Asshown, the process receives (at 7805) a verbal command to startnavigation. The process then sets (at 7810) navigation mode tovoice-activated.

The process then receives (at 7815) a user command through a touch-basedor motion-based input device. The process determines (at 7820) whetherthe lock-screen is active. If not, the process responds to the usercommand (at 7830). The process then exits. In the embodiments whereprocess 7800 is used together with other voice-activated processes, theprocess returns control (at 7830) to other voice-activated processes inorder to respond to the user request. When lock-screen is active, theprocess determines (at 7825) whether navigation is allowed only thoughaudio. If not, the process proceeds to 7830, which was described above.Otherwise, the process optionally makes (at 7835) a short warning sound(e.g., a beep). The process then ignores (at 7840) the user input. Theprocess then exits.

2. Navigation Using Natural Language Utterances

In some embodiments, the user is allowed to request point to pointdirections from the interactive map via a natural language speech query,such as “How do I get from Time Square to the Empire State building?”The interactive map responds to the user's inquiry by providing point topoint directions to the user, for example, either visually and/oraudibly. As the user travels from one location to the next location, theinteractive map optionally (e.g., upon user's verbal request) providesinformation to the user in an audible form, such as time to destination,distance to destination, and current location. In some embodiments, theaudible response from the interactive map is provided to the userwithout deactivating the lock-screen of the user's device.

In some embodiments, the interactive map provides sub-directions as theuser navigates from location to location. The sub-directions areprovided based on the user's current location, a planned route, adestination, and/or the user's request for information. For example,while driving along a route to a predetermined destination, the user mayask the interactive map “What's the building to the right of me?” “Whichway should I go next?” “Where can I get gas?” or “Where can I find anItalian restaurant?” For each of these questions, the interactive mapconsiders the user's current location, the route that the user iscurrently taking, and/or the destination, and provides a contextuallyrelevant response, such as “That was the Ferry building,” “Turn left atthe next corner,” “Here is a list of gas stations near the next fiveexits: . . . ,” or “Here is a list of Italian restaurants near yourdestination: . . . ”

In some embodiments, the interactive map processes various naturallanguage utterances from the user and in response to the utterances,retrieves and presents the user's current navigation status while theuser is traveling along a route. Example navigation status informationincludes information regarding the distance between the user's currentlocation and the user's destination, the estimated time of arrival tothe user's destination, the distance between the user's current locationand the next waypoint (e.g., the next turn, the next exit, or the nextlandmark) along a current or planned route, the estimated time to reachthe next waypoint along a current or planned route, a description of thenext waypoint along the route, a description of the destination, and thelike.

FIG. 79 conceptually illustrates a process 7900 for receiving a naturallanguage utterance and retrieving and presenting the user's currentnavigation status while the user is traveling along a route in someembodiments of the invention. The responses are provided audibly and/orvisually based on the current settings of the user device. As shown, theprocess receives (at 7905) a navigation or map related natural languageutterance. The process then determines (7907) whether the utterance isspecifically related to navigation (e.g., what is my next turn). If not,the process proceeds to 7925, which is described below. Otherwise, whenthe utterance is specifically related to navigation, the processdetermines (at 7910) whether the navigation is going on. Someembodiments set an indicator or flag when navigation starts (e.g., whena destination is selected, search results are found, or navigation isexplicitly started with a commend). Process 7900 utilizes thisnavigation indicator to determine whether navigation is going on. Thisindicator biases voice recognition in some embodiments to compare averbal command with a list of natural languages phrases used fornavigation. Some embodiments support a number of hands-freequestion-answering tasks during navigating in terms of primitives suchas e.g., time remaining, distance remaining, identification of buildingsor objects along the route, location of different services such as gasstations along the route, upcoming navigation directions, how to getsomewhere, questions based on a current dialog context, etc.

When the utterance is related to navigation and navigation is not goingon, the process announces (at 7915) that navigation is not in progress.For instance, in response to “what is my next turn”, the process mightrespond, “no route has been set yet” or “no destination is selectedyet”. The process then ignores (at 7920) the utterance. The process thenends.

When the utterance is just related to the map (e.g., how far is the nextgas station) or when the utterance is related to navigation andnavigation is going on, the process determines (at 7925) whether theutterance relates to distance to the destination, to a waypoint, or anyother distance-based questions relating to navigation or the map. Ifnot, the process proceeds to 7930, which is described below. Otherwise,the process determines (at 7935) the distance between the currentlocation and the destination or waypoint. For instance, a user utterance“How far away am I” and its natural language variations (e.g., “How faraway are we”, “How far do I have to go”, etc.) will cause theinteractive map to retrieve and present the distance-to-destinationinformation based on the user's current location and the location of thedestination.

Similarly, a user utterance “How close is my next turn” and its naturallanguage variations (e.g., “How far away is my next turn”) will causeretrieval and presentation of the distance-to-next-waypoint informationbased on the user's current location and the location of the nextwaypoint on a current or planned route. The process then provides (at7940) the response based on the determined distance. The process thenends.

The process determines (at 7930) whether the utterance relates to timeto the destination, time to a waypoint, or any other time basedquestions related to navigation. If not, the process proceeds to 7955,which is described below. Otherwise, the process determines the inquiredtime based on current time, current location, current speed, speedlimits between the current location and another location, currenttraffic conditions, etc.

For instance, a user utterance “How long do I have to go” and itsnatural language variations (e.g., “When can I get there,” “How close amI,” “How long until I get there,” “How long until we get there,” “Whenwill I get there,” “When do I get there,” “When should I get there,”“When should we be getting there,” “How much longer is it going to be,”“When will I get to [destination name],” etc.) will cause retrieval andpresentation of the time-to-destination information to the user. In someembodiments, the time-to-destination information is determined based onone or more of the current time, the current location, the currentspeed, the speed limits imposed between the current location and thedestination, and the current traffic conditions between the currentlocation and the destination, etc. The process then provides (at 7950)the response based on the determined time. The process then exits.

The process determines (at 7955) whether the utterance is about alocation along the route. If not, the process proceeds to 7960, which isdescribed below. Otherwise, the process provides (at 7965) an answer tothe utterance based on the current location, the destination, or otherpoints along the route. For instance, a user can ask about thedestination or the next waypoint by saying “What's my destination,”“What's next,” “Tell me my next turn,” “Where is my destination,” “Tellme what I have to do,” “Tell me what I have to do next,” and the like.In response, the interactive map provides the information on thedestination or next waypoint (e.g., a description of the destination orwaypoint) based on the user's current location and the destination ornext waypoint on a current or planned route. The process then exits.

The process determines (at 7960) whether the utterance is based on thecurrent dialog between the user and the interactive map. If not, theprocess proceeds to 7975, which is described below. Otherwise, theprocess presents (at 7970) the response based on the current dialog. Theprocess then exits. In some embodiments, the user's utterance isinterpreted based on the current dialogue context. For instance, if theuser has just asked about an earlier waypoint, the user utterance “Whenwill I get there” is interpreted as a request for navigation statusinformation on the next waypoint (e.g., the estimatedtime-to-next-waypoint). In contrast, if the user has just asked aboutthe destination, then the same utterance is interpreted as a request fornavigation status information on the destination (e.g., the estimatedtime-to-destination).

The process determines (at 7975) whether the utterance is based on anyother recognizable navigation or map questions. If yes, the processprovides (at 7980) an answer based on navigational or map informationrelevant to the question. Otherwise, the process exits.

FIG. 80 illustrates a user device 8000 when natural language utterancesare used during voice-activated navigation in some embodiments of theinvention. The figure is shown in three stages 8001-8003. In stage 8001,the user uses the natural language utterance “what is my next turn” (asshown by arrow 8005), to get directions. In stage 8002, the screenoptionally displays the audible interaction between the user and thevoice-activated service. In stage 8003, the voice-activated navigationresponds by providing the audible response (as shown by arrow 8010)“your next turn will be onto 3^(rd) street in 50 feet”. A similar visualdirection 8015 is also displayed on a banner 8020 on the screen.

FIG. 81 illustrates a user device 8100 when natural language utterancesare used during voice-activated navigation in some alternativeembodiments of the invention. The figure is shown in four stages8101-8104. Stage 8101 is similar to stage 8001 of FIG. 80. However, asshown in stages 8102 and 8103, the display is not automatically switchedto show the map again until the user activates a control (such as thebutton 8120). As shown in stage 8104, the map 8130 is displayed once theuser indicates (by activating the control) that the user currently hasno more questions to ask.

FIG. 82 illustrates user device 8000 of FIG. 80 after the user makes aninquiry based on the current dialog. The figure is shown in three stages8201-8203. In stage 8201, the user asks (as shown by arrow 8205) “whenwill I get there”. In stage 8202, the screen optionally displays theaudible interaction between the user and the voice-activated service. Instage 8203, since the current dialog was about the next turn in theroute, the voice-activated navigation responds (as shown by arrow 8210)“in two minutes”. The voice-activated navigation makes the responsebased on the current position, the distance to the next waypoint, thecurrent speed, the traffic condition between the current position andthe next waypoint, etc.

FIG. 83 illustrates user device 8000 when natural language utterancesare used during voice-activated navigation in some alternativeembodiments of the invention. The figure is shown in four stages8301-8304. Stage 8301 is similar to stage 8201 of FIG. 82. However, asshown in stages 8302 and 8303, the display is not automatically switchedto show the map again until the user activates a control (such as thebutton 8320). As shown in stage 8304, the map 8330 is displayed once theuser indicates (by activating the control) that the user currently hasno more questions to ask.

The followings are examples of natural language utterances recognized insome embodiments. One of ordinary skill in the art will realize thatmany other natural language utterances similar to these examples couldbe used to ask navigation related questions.

“How far away am I?”

“How far away are we?”

“How long do I have to go?”

“When is my next turn?”

“How close is my next turn?”

“How long until I get there?”

“How long until we get there?”

“Where is my next turn?”

“When will I get there?”

“When will we get there?”

“When will I get to my destination?”

“When will we get to our destination?”

“When do I get there?”

“When should I get there?”

“When should I be getting there?”

“When should we be getting there?”

“When do I get home?”

“How much longer?”

“How much longer is it going to be?”

“What's next?”

“Tell me my next turn”

“Tell me what I have to do”

“Tell me what I have to do next”

“Tell me when I'm going to get there”

“Tell me when we'll get there”

“Where is my destination?”

“What's the building to the left of me?”

“Which way should I go next?”

“Where can I get gas?”

“Where can I find a hotel?”

3. Voice-Activated Searching and Navigation

In order to facilitate hands-free navigation, some embodiments utilize avoice-activated service to perform user-initiated searches. In someembodiments, this voice-activated service is a part of the interactivenavigation application. In other embodiments, this voice-activatedservice is provided by a voice-activated personal assistant service thatmakes available a wide range of services for the users of the device.Examples of these services are sending messages, placing phone calls,schedule meetings, finding businesses, getting directions, searching theweb, etc., based on user verbal commands and inquiries. An example ofsuch a voice-activated personal assistant is Ski® provided in iPhone®.Some of these embodiments identify one of the items in the search resultand set a route from the current location of the device to theidentified item. The identified search result and the route are thenpresented to the user. The user is provided with an option to navigateto the presented item or to skip to the next item in the search results.

FIG. 84 conceptually illustrates a process 8400 for providingvoice-activated search and navigation in some embodiments of theinvention. FIG. 84 is described with respect to FIGS. 85A-85D. FIGS.85A-85D illustrate 12 stages 8501-8512 of a user interface of someembodiments in which a user is using the voice-activated service tosearch for points of interest and destinations.

As shown in FIG. 84, process 8400 receives (at 8405) a search request.The search request can be made by a verbal command. As shown in stage8501 in FIG. 85A, the user initiates a verbal search request (as shownby arrow 8515). In some embodiments, the voice-activated service isinitiated by activating a button (such as button 8520). In otherembodiments, when the voice level received at the device audio input islouder than a certain threshold (and/or natural language words arerecognized by the device) the voice-activated service is automaticallyactivated.

The process then determines (at 8410) whether navigation is going on.For instance, the process determines whether a destination is alreadyset. If yes, the process retrieves (at 8415) route-aware search results.In some embodiments, the interactive map application maintains routeinformation and shares the remaining route information with process 8400to perform a route-aware search. For instance, in response to “findcoffee shops”, instead of finding coffee shops that are closest to thecurrent location, the process finds coffee shops that are close to thecurrent route even when some of the search results are farther along theroute. The process then proceeds to 8420, which is described below. Inthe example of FIG. 85A, a map 8525 and navigational directions 8537 areshown on the screen. The map identifies the current location 8530 of theuser device and a route 8535 that is currently set for navigation. Forinstance, if a route is set from Los Angeles to Cupertino in Californiaand the user device has moved along the route, stage 8501 shows aportion of the route from Los Angeles to Cupertino in the vicinity ofwhere the device is currently located.

As shown in stage 8502, some embodiments display a transcript 8540 ofthe verbal interactions between the user and the voice-activated serviceto facilitate better communication. Some embodiments (such as theillustrated embodiment) show the transcript as a separate display asshown in stage 8502. Other embodiments (not shown) write the transcripton the same page that was displayed on the foreground when the userstarted the search request (such as the display shown in stage 8501).Also as shown in stage 8502, a navigation banner 8542 is shown on thescreen in order to facilitate navigation along the original route 8535while the voice-activated search is in progress. Although route-awaresearch is described in context of voice-activated search, someembodiments perform route-aware searching when navigation is going onand the user uses touch-based or motion-based input devices.

When the search request is received while navigation is not going on(not shown in FIGS. 85A-85C), process 8400 retrieves (at 8450) thesearch results at the vicinity of the current location of the userdevice (instead of the vicinity of the route as described in operation8415 above). The process then prepares (at 8420) a sequential list ofsearch results. Different embodiments use different criteria for sortingthe list in order to determine which search result is presented to theuser first. For instance, some embodiments use the closest locationfirst. Other embodiments utilize different rankings of each item in thesearch result to sort the list. For instance, a restaurant that has ahigher ranking is shown first. Other embodiments utilize userpreferences either explicitly set or by using the past preferences ofthe user. For instance, a restaurant with lower cost may be presentedfirst.

The process then determines (at 8425) whether any search results areleft in the list. As shown, the process iterates through operations8425-8437 to process each item in the search result list. Therefore, forthe first item in the search result list, the process determines (at8425) whether the search has returned any results. For subsequent itemsin the list, the process determines (at 8425) whether all items in thelist have already been presented to the user. If yes, the processinforms (at 8445) the user that the search has returned no results (forthe first iteration) or that there are no more search results left (forthe subsequent iterations). The process then ends.

Otherwise, when there are more items in the search results list, theprocess sets (at 8430) a route to the next item in the list and presentsthe search result to the user. In order to facilitate hands-freenavigation, the process automatically selects a single route frommultiple routes found and sets the route to the presented search result.As shown in stage 8503 in FIG. 85A, the voice-activated service presentsthe search results to the user in visual (8550) and audible (8552)forms. For instance, the voice-activated service indicates: “I havefound 5 coffee shops in your area. The 1^(st) is Ed's Coffee Shop. Wouldyou like to go there?” In some embodiments, the voice-activated serviceutilizes commonly used abbreviations in the verbal and writtenpresentation to facilitate easy communication.

As shown, the voice-activated service shows a map 8555 that identifiesthe current location of the device 8557, location of the presentedsearch result 8559, and a single route 8558 between the two locations.The screen also shows other useful information 8551 (e.g., the name ofthe presented search result and ratings when available). The user canalso see (or hear) more information about the search result (e.g., bytapping on the banner 8551 or selecting the control 8553 on the banner8551 that shows the search result name or by verbally asking for moredetails about the present search result).

Some embodiments do not show some of the information shown in stage 8503(e.g., the ratings may be displayed only when the user taps on thebanner 8551 or verbally asks for more information). In some embodiments,selecting control 8553 launches a third-party application or opens upthe third-party's website in a browser application that is concurrentlyrunning on the device on which the navigation application is running.For instance, the navigation application of some embodiments launchesthe third-party application (e.g., a Yelp® application) to show the fulltext information, reviews, photos, etc., for the presented searchresult.

Since the user has not decided to navigate to the presented searchresult 8559, the original route 8535 (in this example from Los Angelesto Cupertino as shown in stage 8501) is still being used for actualnavigation and the turn-by-turn navigational direction along route 8535(and not the displayed route 8558) are shown in the navigation banner8542 in stage 8503. Accordingly, the user can continue navigating alongthe original route 8535 while the search results are being presented tothe user by the voice-activated service.

Also, as shown, the map displayed in stage 8503 is different than themap displayed in stage 8501. The map in stage 8501 is a full screenturn-by-turn navigation display (e.g., an optionally 3D map) that showsa portion of the currently navigated route while the map displayed instage 8503 is an overview map that shows the route from the currentlocation of the device to the proposed search result. For instance, theoverview map is a 2D map with the presented search result located closeto the center of the map. In addition, some embodiments display the mapin stage 8503, for example, with different borders, different size,etc., to distinguish between a map displayed by voice-activated serviceto a proposed destination and a map displayed by the interactivenavigation application to a selected destination. In other embodiments,the map displayed in stage 8503 is similar to an overview map (e.g., map7615 shown in FIG. 76). Yet as described further below, in someembodiments, the map is displayed by the interactive navigationapplication (e.g., when requested by the voice-activated service).

Referring back to FIG. 84, the process then determines (at 8435) whetherthe user wants to navigate to the presented search result. Is yes, theprocess proceeds to 8440, which is described below. Otherwise, theprocess determines (at 8437) whether the user wants to terminate search.If yes, the process terminates presenting the search results. In someembodiments, after terminating the voice-activated search, the screendisplays the application that was running in the foreground prior to thebeginning of the search. In some embodiments, the voice-activatedservice returns control to the application that was running in theforeground prior to the start of the search. For example, if theinteractive navigation application was running before the searchstarted, the screen displays navigation information again.

As shown in stage 8504 in FIG. 85B, the user asks the voice-activatedservice to terminate the search (as shown by arrow 8560). In stage 8505,the display optionally shows the transcript 8565 of the verbalcommunication. In stage 8506, the map 8525 that was displayed in stage8501 is displayed back on the screen. The navigational directions 8537,the route 8535, and the current position 8530 of the device are alsorestored. Although the current position 8530 of the device has changeddue to the movement of the device during the search process.Accordingly, the original route 8535 is restored on the screen andnavigation continues to the Cupertino destination since the user did notselect any of the presented search results.

When process 8400 determines that the user does not want to navigate tothe presented search result or to terminate the search, the processproceeds back to 8425 to present the next search result. The processcontinues until (i) the user decides to navigate to a search result,(ii) the user decides to terminate the search, or (iii) there are nomore items to present. For instance, if there are more items in thelist, the process sets a route (at 8430) to the next search result andrepeats operations 8430-8435.

As shown in stage 8507 in FIG. 85C, the user wants to skip “Ed's CoffeeShop” (as shown by arrow 8570). In stage 8508, the transcript 8572 ofthe audible communication is optionally displayed on the screen. Instage 8509, the next item in the search list (in this example “VeniceCucina”) is presented (as shown by arrows 8573 and 8574) to the user.The current location 8530 (which has slightly changed since stage 8501because the device is moving during the search), the location 8575 ofthe new search result, a route 8577 from the current location to thepresented search result, and additional information 8579 about thesearch result are displayed on the screen.

If the user decides to proceed to the presented search result, process8400 of FIG. 84 shows (at 8440) a selected portion of the route andprovides audio and/or visual navigation directions to the selectedsearch result. Although operation 8440 is conceptually shown as a partof process 8400, in some embodiments process 8400 transfers control tothe interactive navigation application in order to provide navigationmap and directions as described throughout this Specification. Theprocess then ends.

As shown in stage 8510 in FIG. 85D, the user decides (as shown by arrow8580) to proceed to the presented search result. As shown, someembodiments also provide a control 8582 that can be selected (e.g., bytapping) in order to select the currently presented search result andproceed to it. In stage 8511, the transcript 8571 of the audiblecommunication is optionally shown on the screen. In stage 8512, thevoice-activated service acknowledges (as shown by arrow 8590) the user'sselection. The full screen turn-by-turn navigation map 8525 and aportion of the route 8587 from the current location 8530 to the selectedsearch result are displayed on the screen. As shown, the map 8525 instage 8512 is similar to the map 8525 in stage 8501 however the route8587 is the route to the selected search result. In some embodiments,control returns to the interactive navigation application to providenavigational directions. In some embodiments, the original route 8535 isalso saved in case the user wants to resume navigating along theoriginal route after visiting the search result (in this example toresume travelling from Los Angeles to Cupertino after visiting VeniceCucina).

FIG. 86 conceptually illustrates an alternative process 8600 forproviding voice-activated search and navigation in some embodiments ofthe invention. In these embodiments, the voice-activated servicedisplays all search results on the display and then identifies them oneat a time in a batch fashion and asks whether the user wants to navigateto the identified search result. As shown, the process receives (at8605) a search request. The search request can be made by a verbalcommand. The process then determines (at 8610) whether navigation isgoing on. For instance, the process determines whether a destination isalready set. If not, the process proceeds to 8645, which is describedbelow.

When navigation is going on, the process retrieves (at 8615) route-awaresearch results. In some embodiments, the interactive map applicationmaintains route information and shares the remaining route informationwith process 8600 to perform a route-aware search. For instance, inresponse to “find coffee shops”, instead of finding coffee shops thatare closest to the current location, the process finds coffee shops thatare close to the current route even when some of the search results arefarther along the route.

In some embodiments when the search is audio-visual (as opposed to e.g.,lock-screen audio-only) the process shows (at 8620) the search resultson a preview display and drops pins at locations of the search results.In some embodiments, the search results are shown either in 3D or 2Ddepending on factors such as the number results found in the search, thelength of the route, etc. Other embodiments switch to a 2D overviewdisplay to show the search results and then switch to 3D display whennavigation starts or continues.

The process also prepares (at 8625) a sequential list of search resultsbased on certain criteria such as proximity to the current location. Theprocess then reads (at 8630) the entries in the list in a batch fashionby cycling through the entries. The process skips or proceeds throughthe list based on verbal commands received from the user. In someembodiments, the interactive map reads a list of information to theuser. For example, when providing the list of gas stations near the nextfive exits, the interactive map reads the names of the gas stations tothe user one by one. The user may skip between items in the list bysaying “Skip” or other trigger words to advance through the list. Whenthe interactive map receives the user's speech input for skipping to thenext item on the list (e.g., a gas station name and related informationsuch as, brand, gas price, distance from the nearest exit, etc.), theinteractive map reads the next item of information in the list orreports that the end of the list has been reached.

FIG. 87 illustrates a user device 8700 during navigation in someembodiments of the invention. As shown, a route 8705 is alreadydetermined on the map 8750 and the current location 8710 of the userdevice is identified on the route 8705. The user starts voice-activatedservice, for example, by pressing a button 8715.

The user then makes a verbal search request (as shown by arrow 8720).The display then optionally shows the transcript 8770 of the verbalcommunication. The search is then performed along the route (as opposedto just in the vicinity of the current location of the user device). Thedisplay transitions to overview 8725 and shows the route with searchresults identified by markers or pins 8730. As shown, the overview 8725shows the search results 8730 and a suggested route to the firstselection (in this example Sam's Coffee Delight). This overview map 8725is different than the navigation map 8750 or an overview for thenavigated route 8705. The overview map 8725 is displayed by thevoice-activated service and shows the search results found based on theuser's verbal search request. The voice-activated service also announces(as shown by arrow 8735) the search results and starts by identifyingthe first search result. In the illustrated embodiment, all searchresults are shown on the map.

FIG. 88 illustrates a user device 8800 during navigation in someembodiments of the invention. FIG. 88 illustrates another embodimentwhere the voice-activated service has received the same search criteriaas in FIG. 87. In the embodiments shown in FIG. 88, however, theoverview map 8825 that shows the markers or pins 8830 is displayed bythe navigation application instead of the voice-activated service.

In other embodiments, the voice-activated service facilitates hands-freenavigation by selecting the first search result and set a route to thesearch result. In these embodiments, a route is displayed to the firstsearch result (e.g., by placing the first search result on the center ofthe map and showing a route from the current location to the firstsearch result). The voice-activated service then gives the name and/orthe description of the first search result and asks whether the userwishes to set the destination to the first search result. If the userwishes to go to the first search result, turn-by-turn navigation to thefirst search result starts. Otherwise, the voice-activated servicecycles through the search results in a batch fashion by selecting thenext search result, setting a route to the next search result, providingthe description of the result to the user, and inquiring whether theuser wishes to go to the provided search result. This process continuesuntil either the user selects a search result or all search results arepresented to the user.

FIG. 89 illustrates the user device 8700 of FIG. 87 when the user doesnot want to select the first coffee shop. As shown, the user makes averbal request to skip the current search item (as shown by arrow 8905).The display then optionally shows the transcript 8920 of the verbalcommunication. The voice-activated navigation then makes an audiblepresentation of the item in the search result (as shown by arrow 8910).This interaction continues until the user selects an item or terminatesthe search. As described above, some embodiments automatically set aroute to the next search result, provide the description of the resultto the user, and inquire whether the user wishes to go to the providedsearch result. In these embodiments, only the next search result (inthis example, Venice Cucina) is displayed on the screen with a routedisplayed from the current location to the search result. If the userselects the search result (e.g., by a verbal command such as “go” or“proceed”), turn-by-turn navigation to the search result starts.Otherwise, the next search result is displayed and the process continuesin a batch fashion.

Referring back to FIG. 86, the process determines (at 8635) whether userhas selected a particular search result. If not, the process ends (or insome embodiments, the process proceeds back to 8630 and continues tocycle through the list until terminated by a user command). Otherwise,the process sets (at 8640) a route to the selected search result. Basedon the user decision, the original route is either saved or is replacedby the route to the selected search result. The process then ends.

When the search request is received while navigation is not going on,the process retrieves (at 8645) the search results at the vicinity ofthe current location of the user device (instead of the vicinity of theroute as described in operation 8615 above). The process then provides(at 8650) the search results in audio and/or visual depending on thecurrent set up. The process then ends. In some embodiments, the processafter retrieving the search results (at 8645) proceeds to 8620, whichwas described above. In these embodiments, search results are presentedto the user as described above by reference to operations 8620-8640instead of operation 8650.

FIGS. 90A-90E conceptually illustrate portions of the voice-activatedservice of some embodiments of the invention that are used during asearch operation. Operations of processes 8400 and 8600 as well asdifferent operations shown in user interfaces in the current “VoiceGuidences” Section are performed by one or more modules in FIGS.90A-90E. One of ordinary skill in the art will recognize that themodules shown in FIGS. 90A-90E are specific to the voice-activatedsearch process of some embodiments and that the voice-activated service,interactive navigation application, and the map service of someembodiments include numerous additional modules (e.g., for map display,route display, additional aspects of navigation, text instructiongeneration, arrow generation, etc.) which are not shown in thesefigures.

The figures show interactions between different modules of thevoice-activated service 9005, map service 9010, and interactivenavigation application 9015 of some embodiments in five stages9001-9005. In some embodiments, the voice-activated service andinteractive navigation application reside on a user device while the mapservice resides outside of the user device. More details of the mapservice of some embodiments are described in the “Map ServiceEnvironment” section, below.

As shown, voice-activated service 9005 includes the following modules:voice input 9020, voice recognition 9025, natural language interpreter9030, display interface 9090, voice to text converter 9035, search listpresenter 9040, search list generator 9045, voice synthesizer 9050, andvoice output 9055. In addition, voice-activated service 9005 includesstorage 9060 for storing a set of navigation and map related naturallanguage utterances. Map service 9010 includes the following modules:map generator 9085, route generator 9065, and search engine 9070. Inaddition, Map service 9010 includes map data storage 9075 and point ofinterest storage 9080. These storages in some embodiments aredistributed and/or include data from several different sources (e.g.,from different vendors, different databases, etc.) Different modules ofthe interactive navigation application 9015 are described throughoutthis Specification and are not shown here for simplicity.

As shown in stage 9001 in FIG. 90A, voice input module 9020 receivessearch requests from the user. For instance, the user starts thevoice-activated service by activating a button or talking louder than athreshold into the device microphone (or an external microphonephysically or wirelessly connected to the device). The voice inputmodule 9020 passes the user's voice request to voice recognition module9025, which converts the voice to words.

Voice recognition module 9025 sends the recognized voice request tovoice to text converter module 9035, which generates a transcript of theaudible communication between the user and the voice-activated service.Display interface 9090 receives the transcript of the communication anddisplays it on the user device.

Natural language interpreter 9030 receives the output of voicerecognition module 9025 and compares the received words with a list ofnatural language phrases (such as the phrases described in “NavigationUsing Natural Language Utterances” section, above) stored in naturallanguage utterances storage 9060. Natural language interpreter 9030module in some embodiments uses heuristics to recognize partial words orpartial phrases that are similar to the recognized natural languageutterances.

In addition, in some embodiments, natural language utterances storage9060 stores navigation related utterances for several differentlanguages. One or more of these sets are used depending on the usersetting of the user device. Natural language interpreter 9030 buildssearch criteria based on recognized navigation related natural languageutterances and sends the criteria to search engine module 9070 of mapservice 9010. The search criteria includes the point of interest orother destination that the user is looking for as well as one or more ofthe current device location, the current route, a search radius, price,ratings, reviews, or other criteria related to the search.

As shown in stage 9002 in FIG. 90B, search engine module 9070 uses mapdata stored in map data storage 9075 and point of interest data storedin point of interest storage 9080 to find results for the given searchcriteria. Search engine module 9070 of map service 9010 sends the searchresult to search list generator module 9045 of voice-activated service9005.

Search list generator module prepares a list (e.g., as described inoperations 8420 or 8625 described above) of the search result. Searchlist presenter module 9040 receives the search list, selects a searchitem, and sends a request to map generator module 9085 of map service9010 for a map and a route from the current device location to thelocation of the search result.

As shown in stage 9003 in FIG. 90C, map generator module 9085communicates with route generator module 9065 and utilizes data from mapdata storage 9075 and point of interest storage 9080 to generate a map(e.g., a map like 8555 in FIG. 85A or 8825 in FIG. 88) and a route to anidentified search result. Search list presenter module 9040 receives theinformation for map and the route and sends the information to displayinterface module 9090 to display on the user device.

Search list presenter module 9040 also prepare a transcript of theaudible presentation for the user and sends a copy to voice synthesizermodule 9050 to generate audible voice and a copy to display interfacemodule 9090 to display on the user device screen. Voice synthesizermodule 9050 synthesizes the voice and sends to voice output module 9055to play on the device speaker(s) or headphones.

As shown in stage 9004 in FIG. 90D, voice input module 9020 receivesuser's (i) selection of a search result, (ii) request to skip thecurrently presented search result, or (iii) request to terminate thesearch. The voice recognition module 9025 receives the user's requestand sends copies of the recognized words to voice to text convertermodule 9035 and natural language interpreter module 9030. Voice to textconverter module sends a transcript of the audible communication todisplay interface module 9090 to display. Natural language interpretermodule 9030 determines the user's request by using the phrases stored innatural language utterance storage 9060 and depending on the type of therequest (i) sends a command to search list presenter 9040 to set a routeto and display the next search result as described above, (ii) sends theidentification of the selected route to interactive navigationapplication 9015, or (iii) terminates the search.

Once the search result selected by the user is identified, interactivenavigation application presents the navigational map and turn-by-turndirections as shown in stage 9005 in FIG. 90E. Interactive navigationapplication as described in this Specification, sends device positioninformation to map service 9010, receives map and navigationinformation, and presents the map and navigation information on the userdevice.

4. Incorporating Navigation into Voice-Activated Service Output

Some embodiments incorporate navigation into voice-activated serviceoutput in order to provide a better user experience. For instance, whenthe user utilizes voice-activated service during navigation, thevoice-activated service incorporates the verbal turn-by-turnnavigational directions into voice-activated service interactions withthe user.

FIG. 91 illustrates 4 stages 9101-9104 of a user interface of someembodiments where navigation is incorporated into voice-activatedservice output. As shown, a map 9125 and navigational directions 9190are shown on the screen. The map identifies the current location 9130 ofthe user device and a route 9135 that is currently set for navigation.In this example, the navigation application provides a verbal guidancewhen the device reaches within 50 feet of the next turn. As shown instage 9101, the user device is still 60 feet from the next turn (asshown by 9190). Therefore, the navigation application is not providingverbal direction guidance.

As shown in stage 9101, the user initiates the voice-activated service(as shown by arrow 9115). In some embodiments, the voice-activatedservice is initiated by activating a button (such as button 9120). Inother embodiments, when the voice level received at the device audioinput is louder than a certain threshold (and/or natural language wordsare recognized by the device) the voice-activated service isautomatically activated. The user in stage 9101 is inquiring about theweather conditions (as shown by 9115), which is not related tonavigation.

As shown in stage 9102, some embodiments display a transcript 9140 ofthe verbal interactions between the user and the voice-activated serviceto facilitate better communication. Some embodiments (such as theillustrated embodiment) show the transcript as a separate display asshown in stage 9102. Other embodiments (not shown) write the transcripton the same page that was displayed on the foreground when the userstarted the search request (such as the display shown in stage 9101).

Also as shown in stage 9102, a navigation banner 9142 is shown on thescreen in order to facilitate navigation along the route 9135. Thisnavigation banner 9142 is narrower than navigation banner 9190. Thenarrower navigation banner is used in some embodiments to shownavigational directions while navigation application is running in thebackground and another application (in this example the voice-activatedservice) is running in the foreground. The navigation banner 9142 showsthat the device has reached within 50 feet of the next turn. Once thedevice is within 50 feet of the next turn, the navigation applicationprepares a verbal voice guidance announcement such as “turn left ontoMain Street in 50 feet”. However, in order not to interfere with thevoice-activated service interactions with the user, the navigationapplication provides the audible output (e.g., in the form of an audiofile or a pointer to an audio file) to the voice-activated service toallow the voice-activated service to make the navigation guidanceannouncement at an appropriate time (e.g., by outputting the receivedaudio file).

As shown in stage 9102, voice-activated service is receiving andtranscribing the verbal user input (user input is shown as phrase 9186to conceptually show that the user is still providing the input or thevoice-activated service is waiting to make sure the user is done makingthe verbal request). The voice-activated service is utilizing voicerecognition to interpret the user's request. If the navigation guidanceis played through the speakers while the user is speaking to thevoice-activated service, the navigation guidance output comes backthrough the microphone and makes it difficult for the voice-activatedservice to recognize what the user actually says. In addition, playingthe navigation guidance might confuse the user (e.g., since the user isexpecting an answer from the voice-activated service).

Once the voice-activated service receives the user input, thevoice-activated service determines whether a navigation guidanceannouncement has to be made. In this example, there is a navigationguidance announcement. As shown in stage 9103, voice-activated serviceinforms the user (as shown by 9180) that there is a navigation directionto announce and proceeds to make the announcement (e.g., by outputtingan audio file received from the navigation application). As shown instage 9104, the voice-activated service provides the response (as shownby 9185) to the user request. Integrating the navigation output into thevoice-activated service output provides a uniform experience for theuser. In some embodiments, the voice-activated service and thenavigation use the same voice synthesizer to make a uniform audiointerface for the user.

FIG. 92 conceptually illustrates a process 9200 used by thevoice-activated service to incorporate navigation output in someembodiments of the application. As shown, the process receives (at 9205)audio information from the navigation application for a navigationaldirection announcement. For instance, as described by reference to FIG.91 above, in stage 9102 the device reaches a point on the route that thenavigation application has to provide a verbal alert to the user.

Process 9200 then determines (at 9210) whether the user is currentlyproviding verbal input to the voice-activated service (e.g., as shown instages 9101 and 9102 of FIG. 91). If so, the process proceeds to 9225,which is described below. Otherwise, the process determines (at 9215)whether the voice-activated service is currently providing an audibleresponse to the user (e.g., as shown in stage 9104 of FIG. 91). If not,the process proceeds to 9230, which is described below. Otherwise, theprocess determines (at 9220) whether the audible response is at a pointthat can be interrupted (e.g., in between sentences). If not, theprocess proceeds to 9225, which is described below.

Otherwise, the process outputs the audio information received from thenavigation application for the navigational direction. The process thenends. In some embodiments, the information received from the navigationapplication is in the form of an audio file that can be played on thedevice audio system. In other embodiments, the information received fromthe navigation application is in the form of text, which is converted tovoice by a voice synthesizer.

When the audible navigation information cannot be played immediately,the process saves (at 9225) the audio information received from thenavigation application (e.g., in memory or storage). The process thenproceeds back to 9210, which was described above. In some embodiments,the process performs operations 9210-9220 after a predetermined delay.In other embodiments, the process automatically checks (e.g., after theuser input is received, after the response to the user is complete,after response to the user reaches a point that can be interrupted,etc.) for any announcements from the navigation application to play.

Although process 9200 was described by reference to announcementreceived by the voice-activated service from the navigation application,some embodiments utilize a similar process to incorporate audibleannouncement from other applications (e.g., when an announcement for anincoming text message has to be made) into the voice-activated serviceoutput to make a better overall experience for the user.

B. Context Aware Voice Guidances

Some embodiments provide context-aware voice guidance. The voiceguidance interacts with other voice services of the user device. Forinstance, the voice guidance does not provide audible guidance while theuser is making a verbal request to any of the voice-activated services.Instead, the voice guidance transcribes its output on the screen whilethe verbal requests from the user are received through the microphone.In some embodiments, when the microphone is turned on (e.g., when thevolume of the input voice is larger than a certain threshold) the voiceguidance determines that the audible guidance has to be suppressed.

In some embodiments, the voice guidance only provides a short warningsound such as a subtle beep as an unobtrusive indicator to get theuser's attention (e.g., at a turn on the route) while the user isspeaking (e.g., on a phone call) or another voice-activated service isproviding audible response to the user's inquires. A navigation banneris displayed on the screen in some embodiments to display the guidanceinformation.

The voice guidance in some embodiments truncates voice guidance the sameway that instructional text is truncated for displaying. The voiceguidance in some embodiments automatically ducks (i.e., reduces thevolume of) music but automatically pauses spoken words of an audio bookin order to provide voice guidance to the user. The voice guidance insome embodiments distinguishes between music that can be ducked andspoken words (e.g., from an audiobook) that the user wants to pauseinstead of being skipped.

The navigation voice volume is adjustable with hardware buttons or touchbuttons while navigating. The standard behavior for audio is to adjustplayback volume only when that audio is playing back. Since voiceguidance is sporadic, short, and somewhat unpredictable, it is verydifficult to adjust the voice guidance volume using the standardmechanism. Furthermore, if the volume is lowered to the point that isinaudible, there would be no way to return the volume to an audiblelevel. So instead of using the standard mechanism, the hardware volumekeys always adjust the voice prompt volume while navigation is inprogress, even when audible navigation directions are not provided.

Some embodiments provide different types of voice prompts depending onwhat other audio sessions are currently active on the device. Althoughseveral examples are described by reference to the navigation system,the same techniques are utilized by some embodiments of the inventionfor providing voice prompts for other services (e.g., announcements forincoming text messages, emails, accessibility assistant prompts, etc.)while other audio services are active on the device. FIG. 93conceptually illustrates a process 9300 that provides different types ofaudible prompts on an electronic device based on the audio sessionscurrently active on the device in some embodiments of the invention.Examples of the audible prompt include a navigation alert provided to auser during navigation, an alert indicating that a text message isreceived, an alert indicating that an email is received, etc. Examplesof active audio sessions are ongoing phone calls, interactions ofpersonal assistants (such as Siri®) with a user, video calls (such asFaceTime®), voice over IP (VoIP) sessions, playing music, podcasts orother spoken words, etc.

In some embodiments, the device at system level (e.g., the device audiosystem or the device operating system) is aware of different types ofaudio sessions that are active on the device at each time and whetheraudio is being played or received by different device services. Someembodiments categorize and define the type of audio being played orreceived. The audio categories and the information about the categoriesof active audio sessions are maintained by the audio system.

Examples of different active audio sessions include a phone call or avideo call in progress using the device phone service, a VoIP serviceprovided by the device, audio recording in progress by the device, thedevice voice-activated personal assistance service being active, musicbeing output by the device, a podcast in progress, an audio book beingread to the user, etc. Any application (such as navigation, textmessaging, email, etc.) that requires providing audio alerts inquiresthe system about the type of the prompt that can be played at aparticular time. The audio system utilizes information regarding thetype of the audio sessions that are currently active on the device andinforms the application about the type of voice prompt that cancurrently be played. The application then provides an appropriate audiofile to the audio system to play.

As shown, process 9300 receives (at 9305) an inquiry about the type ofthe voice prompt that can currently be played on the device. Forinstance, the process receives the inquiry from an application such asthe navigation application that needs to play an alert to the user toturn left onto Main Street in 50 feet. The process then determines (at9310) whether an audio session is active where an audio service iscognitively listening to the user voice. For instance the processdetermines whether a voice recognition service or an explicit audio filerecording is currently receiving voice input from the user. If not, theprocess proceeds to 9320, which is described below.

Otherwise, when a voice recognition service or an explicit audio filerecording is currently receiving voice input from the user, the processinforms (at 9315) the inquiring application that no voice prompts areallowed at this time. The process then exits. In some embodiments,process 9300 not only informs an inquiring application that no promptsshould be played but also enforces across the system that if anapplication attempt to play a prompt during this time (e.g., fromnavigation application or any other application), the prompt issuppressed at the system level to ensure that no incidental audiointerferes with user audio input where quality is paramount (e.g., whenthe personal assistant is receiving the user's verbal input or a voicememo is being recorded). In some embodiments, when audio prompts are notallowed, the inquiring device provides other types of alerts (e.g.,provides a transcript of the audio prompt on the display of the device).

The audio sessions that are not interrupted by voice prompts include thesessions where a device service is cognitively receiving input from theuser and the quality of the audio input is important. An example of avoice recognition service that is currently receiving voice input iswhen a personal assistant service listening to the user. For instance,the user is asking for the weather conditions in a particular city. If avoice prompt (e.g., a navigation prompt to alert the user to turn leftin 50 feet) is played while the user is speaking to the voicerecognition service, the prompt might confuse the user. In addition,when the prompt is played through the speakers, the output comes backthrough the microphone and makes it difficult for the voice recognitionservice to recognize what the user actually says.

An example of an explicit audio file recording is when the user iscreating a voice memo and the user's voice is actively being recorded.Whenever the quality of the voice input is important (e.g., when a voicerecognition service is listening to the user or a voice memo is beingrecorded), it is desirable to suppress voice prompts not to include anavigation voice prompt in the middle of a recorded voice memo.

The process determines (at 9320) whether a conversational audio sessionwith spoken words (other than the services described in operation 9310above) is currently active. The conversational audio sessions include aphone call in progress, a video call in progress, an active VoIPsession, interactions of the user with the voice-activated personalassistant where the personal assistant has already received the user'srequest and is either providing the answer or is in the process offinding the answer, etc. This type of conversational audio sessionincludes sessions where the user is involved in a two-way interaction(e.g., on the phone or with the voice-activated personal assistant)where a short non-verbal interrupt can be tolerated (as opposed to avoice memo or a user command to a voice recognition service where nointerruptions are desirable).

When a conversational audio session with spoken words is currentlyactive, the process proceeds to 9325, which is described below.Otherwise, the process informs (at 9330) the inquiring application thata long verbal prompt can be played. The process then optionally receives(at 9335) the audio file or audio signal for the verbal voice prompt toplay. The process then plays (at 9340) the verbal voice prompt. Theprocess then exits. An example of long verbal prompt is a navigationprompt alerting the user to turn right onto Wilshire Boulevard in 100feet. This long verbal prompt is used when audio session that iscurrently active does not require user's verbal input (e.g., when noaudio is played, when non-spoken words audio such as music is beingplayed, when a spoken word podcast is being played, etc.).

When a conversational audio with spoken words is active, the processinforms (at 9325) the inquiring application that a short non-verbalprompt can currently be played. The process then optionally receives (at9345) the audio file or audio signal for the short non-verbal voiceprompt to play. The process then plays (at 9350) the non-verbal voiceprompt. The process then exits. Examples of spoken words in thiscategory are phone conversation using the device or when avoice-activated personal assistant has received the user input and isfinding the results. For instance, while the personal assistant ischecking the weather condition in a particular city based on the userinput, a short non-verbal navigation prompt is played to alert the userof an upcoming turn. As described further below, some embodimentsutilize panning, tone loudness, and/or different sequences of tones toprovide directional information in a non-verbal prompt.

Some embodiments provide another context aware interaction between avoice prompt and the audio that is played by an active audio session. Inthese embodiments, the active audio is played based on the currentvolume settings of the device sound system (e.g., the device speakers)or the sound system to which the device is connected (e.g., the carstereo system). The application that is providing the audio prompt inthese embodiments has a separate relative volume setting. For instancethe device navigation application has the relative volume settings ofloud volume, normal volume, low volume, or no voice in some embodiments.The voice prompt and/or the concurrent audio are played at certainrelative volume levels such that the user perceives the navigationprompt and the concurrent audio levels to be at the same level. Therelative volume levels are determined either based on the measurementsmade on the audio being played or based on the type of the audio beingplayed.

FIG. 94 conceptually illustrates a process 9400 for providing audibleprompts while another audio session is active in some embodiments of theinvention. As shown, the process receives (at 9405) a request to make anaudible voice prompt. For instance, the process receives the requestwhen process 9300 determines that either a long or a short audibleprompt can be played at the current time. The process then determines(at 9410) whether signal measurements are available to determine theloudness of the content currently being played. If not, the processproceeds to 9415, which is described below.

When signal measurements are available, the process dynamically assesseshow the user perceives the volume of the content being played. Forinstance, some embodiments provide measurements for the power and otherproperties of the audio signals. Based on the measurements as well asthe relative volume settings for the particular application that isproviding the prompt, the process plays (at 9420) the voice prompt andthe concurrent audio at volume levels such that the user perceives thenavigation prompt and the concurrent audio levels to be at the samelevel. The process then exits.

When signal measurements are not available, the process plays (at 9415)the voice prompt and the concurrent audio based on the type of the audiobeing played as well as the relative volume settings for the particularapplication that is providing the prompt. The voice prompt and theconcurrent audio content are played at volume levels such that the userperceives the navigation prompt and the concurrent audio levels to be atthe same level. The process then exits. Details of the operations ofprocess 9400 are described further below by reference to FIG. 101 andprocess 10100. Some embodiments utilize a combination of processes 9300and 9400 to provide (or to suppress) voice prompts.

1. Providing Different Types of Audible Prompts Based on the Type of theActive Audio Session

FIG. 95 conceptually illustrates a more detailed process 9500 forproviding different types of audible prompts on an electronic device insome embodiments of the invention. As shown, the process receives (at9505) a request to make a voice announcement. Examples of such voiceannouncement were described by reference to operation 9305, above. Theprocess then determines (at 9510) whether a voice recognition service iscurrently receiving voice input. If yes, the process proceeds to 9525,which is described below. Otherwise, the process determines (at 9515)whether a voice memo is currently being recorded. Examples of a voicerecognition service that is currently receiving voice input and a voicememo currently being recorded are provided by reference to operation9315, above. If yes, the process proceeds to 9525, which is describedbelow.

Otherwise, the process determines (at 9520) whether an audio sessionwith spoken words is currently active. This type of audio sessionincludes on-going phone conversions, video calls, and VoIP sessions.This type of audio session also includes a voice-activated personalassistant session where the voice-activated personal assistance hasalready received the user input and is now either talking back to theuser or is in the process of searching to find the answer to the userrequest. If not, the process proceeds to 9545, which is described below.

a) Providing Directional Information Using Non-Verbal Audible Prompts

When an audio session with spoken words is currently active, process9500 provides (at 9535) a non-verbal prompt. The process also optionallydisplays (at 9540) the announcement. The process then exits. Thenon-verbal prompt in some embodiments (such as the non-verbal promptsduring navigation) provide directionality information. FIG. 96conceptually illustrates a process 9600 for providing non-verbal promptswith directionality information (e.g., during navigation) in someembodiments of the invention. Process 9600 is activated, e.g., duringoperation 9535 of process 9500 when an active audio session is inprogress and a determination is made to make a non-verbal audible promptto provide directionality information. As shown, process 9600 receives(at 9605) a request to make a non-verbal voice prompt that includesdirectionality information (e.g., to alert the user to change directionsduring navigation).

Process 9600 then determines (at 9610) whether it is possible to providethe prompt in stereo. For instance, if the currently active audiosession is stereo (e.g., a voice-activated personal assistance session)then it is possible to provide the non-verbal prompt in stereo. On theother hand if a mono audio session (e.g., a mono phone call) is active,it is not possible to provide the prompt in stereo. When the activeaudio session is in stereo, the process proceeds to 9620, which isdescribed below.

Otherwise, the process makes (at 9615) the non-verbal voice prompt byusing a set of tones that provide directionality by utilizing differenttone sequences and/or different tone loudness to correspond to differentdirections required by navigation. FIG. 97 illustrates a user device9700 when a navigation session and a phone call are in progress in someembodiments of the invention. The figure is shown in three stages9701-9703. In stage 9701 a phone conversation (as shown by the phraseslabeled 1 and 2) is in progress between the device user, who has usedthe device to make a phone call, and another person whose voice is beingplayed through a single speaker of the device. In addition, a navigationsession is in progress (as shown by the route 9710) and the user deviceis still 500 feet from the next turn (as shown by navigational direction9705).

In stage 9702, the phone conversation is still in progress (as shown bythe phrases labeled 3 and 4). The device in this stage has reached to apoint (as shown by navigational direction 9715 and the location 9720 ofthe device on the route 9710) where the navigation application has toprovide a voice prompt to alert the user to make a right turn in 50feet. However, since the phone call is in progress, instead of a verbalannouncement such as “turn right onto Freedom Street in 50 feet”, thenavigation application provides a non-verbal prompt. As shown by thephrase labeled 5, a voice prompt (ding dong in this example) is played.

In stage 9703, the phone conversation is still in progress (as shown bythe phrases labeled 6 and 8). The device in this stage has reached thenext turning point (as shown by navigational direction 9725 and thelocation 9730 of the device along the route) where the navigationapplication has to provide a voice prompt to alert the user to make aleft turn in 50 feet. Since the turning is to the left, this time adifferent sequence of tones (dong ding in this example) are played. Byusing different sequences of tones (ding dong versus dong ding), theuser is provided with directional information required for propernavigation. In other embodiments, instead of changing the sequence oftones, different tones are used for different directions. For instance,ding (or ding ding) for turning right and dong (or dong dong) forturning left or vice versa.

FIG. 98 illustrates another example of providing directional informationby non-verbal prompts in some embodiments of the invention. The figureillustrates a user device 9800 when a navigation session and a phonecall are in progress in some embodiments of the invention. Forsimplicity, the same scenario for the phone conversation andnavigational directions as FIG. 97 is shown in FIG. 98. As shown instage 9802, the navigation application provides the non-verbal prompt toalert the user to make a right turn. The prompt includes a loud tone(ding, as conceptually shown by the larger characters in the phraselabeled 5) and a soft tone (dong, as conceptually shown by the smallercharacters in the phrase labeled 5).

In stage 9803, since the next turning point is a left turn, the promptincludes the same tones as for the right turn. The first tone, however,is a soft tone (ding, as conceptually shown by the smaller characters inthe phrase labeled 5) and a loud tone (dong, as conceptually shown bythe larger characters in the phrase labeled 5). By using the same set oftones with different loudness, the user is provided with directionalinformation required for proper navigation. In other embodiments thesame tone or the same sequence of tones are used with different loudnessto show directions (e.g., a loud ding for turning left and a soft dingfor turning right or vice versa).

Referring back to FIG. 96, when the active audio session is in stereo,the process pans the voice prompt to show directions. In someembodiments, in addition to panning, the process optionally uses (at9620) a set of non-verbal tones that provide directionality by utilizingdifferent tone sequences and/or different tone loudness to correspond todifferent directions. Examples of non-verbal tones to use direction weredescribed above by reference to FIGS. 97 and 98. The process then makes(at 9625) the non-verbal voice prompt by panning the non-verbal tone ortones in the direction that corresponds to the change of directionrequired for navigation. The process then exits.

FIG. 99 illustrates an example of providing directional information bypanning non-verbal prompts in some embodiments of the invention. Thefigure illustrates a user device 9900 in two stages 9901 and 9902. Inthis example, the user device 9900 is connected to car speakers 9905(e.g., through an auxiliary line-in of the car audio system or by usingBluetooth®). The same example applies when the user device has stereospeaker.

As shown in stage 9901, the user has activated the voice-activatedpersonal assistant (e.g., by activating control 9910) or speaking withloudness higher than a pre-determined threshold. The voice-activatedpersonal assistant is asking (as shown by the phrase labeled 1) the user“what can I help you with?” The user is inquiring (as shown by thephrase labeled 2) about the weather conditions in Los Angeles.

In stage 9902, while the voice-activated personal assistant is searchingfor the weather conditions, the device approaches the next turn fornavigation (as shown by 9910). As shown, the non-verbal voice prompt ispanned to the right speaker to alert the user to next navigation turnand provide directional information for turning right. Similarly, thevoice prompt is panned to the left speaker when the next navigationaction is a left turn. As described above, some embodiments combinepanning to particular sequence or loudness of tones to provideadditional directional information.

b) Providing Verbal Prompts at a Volume Level to be Perceived at theLevel of Media Content Currently being Played

Referring back to FIG. 95, process 9500 determines (at 9545) whetheraudio such as music or spoken words that do not involve two-way userparticipation is currently being played by the device. An example ofspoken words that do not involve user participation is playing a podcastthat includes spoken words. If yes, the process proceeds to 9550, whichis described below. Otherwise, when no sounds are played (i.e., whenthere is silence) the process plays (at 9560) a verbal announcement witha volume that is determined based on the current relative volumesettings for the application that is providing the prompt and/or at alevel that is determined to be best perceived by the user. The processthen exits.

When audio such as music or spoken words that do not involve two-wayuser participation is currently being played, the process optionallyducks (at 9550) the audio sounds that are currently being played. Theprocess then plays (at 9555) a verbal announcement with a volume that isdetermined based on the type of the sounds currently being played, thecurrent relative volume setting for the application that is providingthe prompt, and/or at a level that is determined to be best perceived bythe user. The process then exits. Details of operations 9550 and 9555are described by reference to process 10100 described in FIG. 101,below.

Some embodiments provide user selectable relative volume level settingsfor some applications such as the navigation application. FIG. 100illustrates a user interface 10000 for setting relative prompt levels ofthe navigation application in some embodiments of the invention. Theapplication, referred to as Maps 10005 in this example has fouruser-selectable relative volume settings. The settings are no voice10010, low volume 10015, normal volume 10020, and loud volume 10025.

These volume settings 10010-10025 are used to play the navigationprompts when there is no other audio concurrently played on the device.The no voice 10010 setting suppresses the navigation audio prompts. Insome embodiments, the other volume settings 10015-10025 are used asguidelines to adjust the navigation prompt voice level while anotheraudio is concurrently played on the device in order for the user toperceive the navigation prompt and the concurrently played prompt at thesame relative volume level.

FIG. 101 conceptually illustrates a process 10100 for providing verbalnavigation prompts while another audio session is active in someembodiments of the invention. Process 10100 is also utilized forproviding other type of prompts (e.g., incoming text message alerts) insome embodiments. As shown, the process receives (at 10105) a request tomake a verbal voice prompt. For instance, the process receives therequest when process 9300 determines that either a long or a shortaudible prompt can be played at the current time. Process 10100 thendetermines (at 10110) whether signal measurements are available todetermine the loudness of the content currently being played. If not,the process proceeds to 10115, which is described below.

When signal measurements are available, the process dynamically assesses(at 10120) how the user perceives the volume of the content beingplayed. For instance, some embodiments provide measurements for thepower and other properties of the audio signals. These measurements areutilized to determine how loud the user perceives the signal that isbeing played by the active audio session. The process also optionallyassesses (at 10125) other properties of the audio system such as therelative volume settings of the navigation application (e.g., asdescribed above by reference to FIG. 100), the audio route, the outputdevice (e.g., device speaker, headphone, car speakers, etc.) and thecurrent audio system volume settings. Based on all of these assessments,the process plays (at 10130) the voice prompt and the concurrent audioat volume levels such that the user perceives the navigation prompt andthe concurrent audio levels to be at the same level. The process thenbrings (at 10137) the volume level of the concurrent audio back to thepre-prompt level if the volume was ducked to play the prompt. Theprocess then exits.

When signal measurements are not available, the process assesses (at10115) the perceived level of the currently played content based on thetype of the content. Examples of this assessment are described below.The process also optionally assesses (at 10135) other properties of theaudio system such as the relative volume settings of the navigationapplication, the audio route, the output device (e.g., device speaker,headphone, car speakers, etc.) and the current audio system volumesettings. The process then optionally ducks (at 10140) the contentcurrently being played based on the assessments. The process then, basedon the assessments, plays (at 10145) the voice prompt at a volume levelsuch that the user perceives the navigation prompt volume at the samelevel as the volume level of the content currently being played. Theprocess then brings (at 10150) the volume level of the content back topre-prompt level if the volume was ducked to play the prompt. Theprocess then exits.

FIG. 102 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. In thisexample, the navigation prompt relative volume level is set at high(i.e., at loud volume 10025 as shown in FIG. 100) and the media beingplayed is of a type (such as music) that is inherently loud. The voiceprompt, on the other hand is not as loud as the recorded music. In orderfor the user to perceive the prompt at the level of the media content,the voice prompt is played at a higher volume level to create a highcontrast between the prompt and the media content.

As shown in FIG. 102, the media content 10205 is ducked. The voiceprompt 10210 is then played between times t₀ 10215 and t₁ 10220. Themedia level is then brought back to the pre-prompt volume level. In thisexample, the content was played at 70% of the maximum hardware volumesetting prior to the prompt. The media content is ducked to 20% of itsvolume (i.e. 14% of the hardware volume level). The prompt is thenplayed at the level that the media content was played prior to theprompt. An inherently loud audio type (such as music) is thereforeducked and the relatively soft prompt is played at a higher relativevolume level to make the user perceive the media content and the promptat the same volume level.

Although specific volume levels and specific type of media being played(inherently loud) are used in the example of FIG. 102, one of ordinaryskill in the art will realize that ducking the media content volumelevel to a lower level and playing the voice prompt at the level thatthe media content was played prior to being ducked can be used withother volume level settings and other type of media being played (otherthan media inherently being loud) when the navigation prompt relativevolume level is set at high.

FIG. 103 illustrates providing a voice prompt on a device while therelative volume level of the voice prompt is set at loud volume 10025(as shown in FIG. 100) and the content that is inherently loud is beingplayed in some embodiments of the invention. The device 10300 is shownin two stages 10301 and 10302. In stage 10301, music is played (as shownby 10305) on the device while navigation is in progress (as shown by theroute 10320).

In stage 10302 the device has approached the next turn (as shown bynavigational direction 10310 and the location 10315 of the device alongthe route 10320). As described by reference to FIG. 102 above, thevolume level of the media content is ducked (as conceptually illustratedby smaller tones) while the voice prompt is played at the same levelthat the media content was being played prior to the prompt. Since theprompt is softer than the music, the user perceives the two volumes atthe same level.

FIG. 104 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. In thisexample, the relative prompt volume level is set to low (i.e., to lowvolume 10015 as shown in FIG. 100) and the media being played is of atype (such as a podcast) that is inherently soft. The voice prompt, onthe other hand is not as soft as the spoken words of the podcast. Inorder for the user to perceive the prompt at the level of the mediacontent, the voice prompt is played at a volume level comparable orslightly higher than the media content.

As shown in FIG. 104, the media content 10405 is ducked. The voiceprompt 10410 is then played between times t₀ 10415 and t₁ 10420. Themedia level is then brought back to the pre-prompt volume level. In thisexample, the content was played at 70% of the maximum hardware volumesetting. The media content is ducked to 20% of its volume (i.e., 14% ofthe hardware volume level). The prompt is then played at 30% of thelevel that the media content was played prior to the prompt (i.e., at21% of hardware volume level). An inherently soft audio type (such aspodcast) is therefore ducked and the prompt is played at a same orslightly higher level. Since the podcast is softer than the voiceprompt, playing the podcast at 20% and the voice prompt at 30% of thepre-prompt podcast volume level makes the user to perceive the twovolumes at the same level and also creates enough contrast for the userto hear the voice prompt while still being able to cognitively listen tothe podcast.

Although specific volume levels and specific types of media being played(inherently soft) are used in the example of FIG. 104, one of ordinaryskill in the art will realize that ducking the media content volumelevel to a lower level and playing the voice prompt at the same orslightly higher level that the media content is played after beingducked can be used with other volume level settings and other types ofmedia being played (other than media that is inherently being soft) whenthe navigation prompt relative volume level is set to low.

FIG. 105 illustrates providing a voice prompt on a device when therelative prompt volume level is set to low (i.e., to low volume 10015 asshown in FIG. 100) and the content that is inherently soft is beingplayed in some embodiments of the invention. The device 10500 is shownin two stages 10501 and 10502. In stage 10501, a podcast is played (asshown by the phrase labeled 1) on the device while navigation is inprogress (as shown by the route 10520).

In stage 10502 the device has approached the next turn (as shown bynavigational direction 10510 and the location 10515 of the device alongthe route 10520). As described by reference to FIG. 104 above, thevolume level of the media content is ducked (as conceptually shown bysmaller characters for the phrase labeled 2) while the voice prompt isplayed (as shown by the phrase labeled 3) at the same or a slightlyhigher volume level as the ducked media content.

FIG. 106 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. This isanother example where the media being played is of a type (such as apodcast) that is inherently soft. The voice prompt, on the other hand isnot as soft as the spoken words of the podcast. In this example, themedia content 10605 continues to be played at the same level while thevoice prompt 10610 is being played.

The voice prompt 10610 is played between times to 10615 and t₁ 10620. Inthis example, the content was played at 70% of the maximum hardwarevolume setting. The prompt is then played at 60% of the level that themedia content is played prior to and during the voice prompt (i.e., at42% of hardware volume level). Since the podcast, is inherently softerthat the voice prompt, playing the podcast at 100% and the voice promptat 60% of the pre-prompt podcast volume level makes the user to perceivethe voice prompt and the media content at the same volume level and alsocreates enough contrast for the user to hear the voice prompt whilestill being able to cognitively listen to the podcast.

Although specific volume levels and specific type of media being played(inherently soft) are used in the example of FIG. 106, one of ordinaryskill in the art will realize that continuing to play the media contentat the same level and playing the voice prompt at a lower level than themedia content is played can be used with other volume level settings andother types of media being played (other than media that is inherentlysoft) when the navigation prompt's relative volume level is set to low.

FIG. 107 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. In thisexample, the relative volume level of the prompt is set to normal (i.e.to volume normal 10020 as shown in FIG. 100) and the media being playedis of a type (such as an audio book) that is recorded at a normal level.

As shown in FIG. 107, the media content 10705 is ducked. The voiceprompt 10710 is then played between times t₀ 10715 and t₁ 10720. Themedia level is then brought back to the pre-prompt volume level. In thisexample, the content was played at 70% of the maximum hardware volumesetting. The media content is ducked to 20% of its volume (i.e., 14% ofthe hardware volume level). The prompt is then played at 80% of thelevel that the media content was played prior to the prompt (i.e., at56% of hardware volume level). Since the voice prompt and the audiocontent were recorded at comparable loudness levels, the prompt playedat 80% and the media content at 20% of the pre-prompt media contentvolume level makes the user to perceive the prompt and the audio contentat the same volume level and creates enough contrast for the user tohear the voice prompt while still being able to cognitively listen tothe audio book.

Although specific volume levels and specific type of media being played(media recorded at a normal level) are used in the example of FIG. 107,one of ordinary skill in the art will realize that ducking the mediacontent volume level to a lower level and playing the voice prompt at alevel between the ducked level of the media content and the level thatthe media content was played prior to being ducked can be used withother volume level settings and other types of media being played whenthe navigation prompt's relative volume level is set to normal.

FIG. 108 illustrates providing a voice prompt on a device where therelative volume level of the prompt is set to normal (i.e. to volumenormal 10020 as shown in FIG. 100) and the content is recorded at asimilar loudness level as the voice prompt in some embodiments of theinvention. The device 10800 is shown in two stages 10801 and 10802. Instage 10801, an audio book is played (as shown by the phrase labeled 1)on the device while navigation is in progress (as shown by the route10820).

In stage 10802 the device has approached the next turn (as shown bynavigational direction 10810 and the location 10815 of the device alongthe route 10820). As described by reference to FIG. 107 above, thevolume level of the media content is ducked (as conceptually shown bysmaller characters for phrase labeled 2) while the voice prompt isplayed (as shown by the phrase labeled 3) at a level (e.g., at 80%) ofthe pre-prompt audio book volume level to create the perception for theuser that the audio prompt and the audio book are played at the samevolume level while being able to cognitively listen to the audio book atthe same time.

c) Suppressing Voice Prompts

Referring back to FIG. 95, when a voice recognition service (e.g., thevoice-activated personal assistant or voice-activated service) iscurrently receiving voice input from the user or user's voice is beingrecorded to create a voice memo audio file, the process suppresses (at9525) the voice prompt. The process optionally displays (9530) ortranscribes the voice announcements. The process then exits.

In some embodiments, the process stores the voice prompts and checks ata pre-determined later time whether the prompt is still relevant andwhether it can be played. For instance, some embodiments duringnavigation calculate the estimated time to the upcoming turn and retryoperations 9510 and 9515 at a pre-determined time prior to the devicereaching the turning point to determine whether the prompt can beplayed. For instance, a full verbal prompt can be played if thenavigation point is not reached yet and the user has ended recording thevoice memo or the voice-activated personal assistant. On the other hand,a short non-verbal prompt can be played if the navigation point is notreached yet and the user has ended a verbal request to the personalassistant and the personal assistant is currently finding or playing theresponse.

2. Alternative Embodiments for Providing Context Aware Audible Prompts

Some embodiments provide additional techniques to enhance the userexperience when a voice prompt is played while another audio session isactive. As described below different embodiments control the time gapbetween ducking the media content and playing the voice prompt in orderto control the contrast between the voice prompt and the media content.Other embodiments pause the media content (instead of ducking) whenpausing the content is possible.

a) Controlling the Gap Between the Audio Prompt and Media Content

FIG. 109 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. In thisexample, the media content 10905 is ducked at t₀ 10915 but the voiceprompt 10910 is not played until time t₁ 10920. The gap 10925 istherefore created to make a longer pause before the prompt is played. Amore pronounced pause between the media content and the voice prompt(e.g., half a second delay) heightens the user awareness for theupcoming prompt. Although the voice prompt is shown at the level thatthe content was being played prior to the prompt, a person of ordinaryskill in the art will realize that the prompt can be played at any levelwhile a gap is created as shown in FIG. 109.

FIG. 110 conceptually illustrates the volume level of audio played on auser device versus time in some embodiments of the invention. In thisalternative embodiment, the prompt is played at the moment the mediacontent is being ducked to minimize the period over which the content isducked. Alternatively, the prompt can start playing while the media isbeing ducked (i.e., between to 11015 and t₁ 11020).

b) Pausing the Media Content

FIG. 111 conceptually illustrates a process 11100 for playing a voiceprompt in some embodiments of the invention while another audio sessionis active. As shown, the process receives (at 11105) a request to make averbal voice prompt. The process then determines (at 11110) whether thecontent being played is of a type that can be paused. Similar to theaudio types that allow long, short, or no prompts, some audio typesallow the playback to be paused. For instance, an audio book (which isplayed and controlled by the device) can be paused while a phoneconversation cannot be paused.

When the content cannot be paused, the process optionally ducks (at11130) the content. The process then provides (at 11135) the voiceprompt. The process then resumes (at 11140) playing the content fromwhere it has reached after making the voice prompt. Details ofoperations 11130-11140 are similar to operations described by referenceto FIG. 95, above and are not repeated here for brevity.

On the other hand, when the content playback can be paused, the processpauses (11115) the content. The process then provides (at 11120) thevoice prompt (e.g., a full verbal voice prompt or a non-verbal shortprompt). The process then resumes (at 11125) playing the content fromwhere the content was paused.

FIG. 112 illustrates providing a voice prompt on a device while contentthat can be paused is being played in some embodiments of the invention.The device 11200 is shown in three stages 11201-11203. In stage 11201,an audio book is played (as shown by the phrase labeled 1) on the devicewhile navigation is in progress (as shown by the route 11220).

In stage 11202 the device has approached the next turn (as shown bynavigational direction 11210 and the location 11215 of the device alongthe route 11220). Instead of ducking the audio book, the audio bookplayback is paused and the voice prompt is played (as shown by thephrase labeled 2). In stage 11203, the voice prompt playback is completeand the audio book play is resumed (as shown by the phrase labeled 3).

3. Software Architecture

FIG. 113 conceptually illustrates the high-level software architectureof the context aware guidance system of some embodiments. One ofordinary skill in the art would recognize that the modules shown in FIG.113 are specific to the context aware guidance system of someembodiments and that the audio service and interactive navigationapplication of some embodiments include numerous additional modules(e.g., for map display, route display, additional aspects of navigation,text instruction generation, arrow generation, audio input processing,etc.) which are not shown in this figure.

As shown, the interactive navigation application 11310 includes thefollowing modules: “provide prompt playing volume level” 11315, “provideaudio file for prompts” 11320, and “generate navigational directions”11325. In addition, interactive navigation system 11310 includes datastorages 11330 and 11335 for storing audio files for different promptsand relative audio settings for navigation respectively.

The audio system 11305 includes the following modules: “determineallowed prompt types” 11340, “set audio levels” 11345, “set pan level”11350, and “audio output” 11355. In addition, audio system 11305includes data storages 11360 and 11365 for the currently active audioservices and audio types and their corresponding allowed promptsrespectively.

As shown, “determine allowed prompt types” module 11340 receives theinformation regarding the currently active audio services from thecurrently active audio services storage. The “determine allowed prompttypes” module 11340 also receives the list of the audio types and theircorresponding allowed prompts from audio types and allowed promptsstorage 11365. Based on the type of the active audio services,“determine allowed prompt types” module 11340 determines the audioprompt types that are currently allowed. For instance, if the currentactive audio service is the voice-activated personal assistant, which iscurrently providing answer to a user's request, short non-verbal audioprompts are currently allowed. Also, in the event that no prompts areallowed, the audio system 11305 not only informs an inquiringapplication (such as the interactive navigation application 11310) thatno prompts should be played but also enforces across the system that ifan application attempt to play a prompt during this time (e.g., fromnavigation application or any other application), the prompt issuppressed at the system level to ensure that no incidental audiointerferes with user audio input where quality is paramount.

The “generate navigational directions” module 11325 sends navigationdirections to announce to “provide audio file for prompts” module 11320.The “provide audio file for prompts” module 11320 receives the prompttypes that are currently allowed (e.g., when no other audio is beingplayed both short non-verbal prompts and long verbal prompts areallowed). The “provide prompt playing volume level” module 11315receives the current volume settings for navigation from the relativeaudio setting for navigation storage 11335, determines the promptplaying volume and forwards it to the “provide audio file for prompts”module 11320.

Based on the received information, the “provide audio file for prompts”module 11320 either generate an audio file for the audio prompts orretrieves an audio file for the prompt from the audio files for promptstorage 11330. The “set audio levels” module 11345 receives audio filefor the prompt, the prompt directionality (when the prompt is anon-verbal prompt) and prompt playing volume from “provide audio filefor prompts” module 11320. The “set audio levels” module 11345 alsoreceives audio content to play from the currently active audio service11370.

The “set audio levels” module 11345 determines the volume level for theaudio content and sends the volume level along with the volume level forthe prompt, the audio file for the prompt, and the concurrently playedaudio content to “audio output” module 11355. The “set audio levels”module 11345 also sends the prompt directionality (when the prompt is anon-verbal prompt) to “set pan level” module 11350 which sets the panlevel and sets the pan level to “audio output” module 11355, whichoutputs the audio content and the audio prompts. In the embodimentsillustrated in FIG. 113, the audio system 11305 (and the “set pan level”11350 module) is described as being responsible for crating the stereopan for the directionality of the prompt. In other embodiments (notshown), the interactive navigation application 11310 determines (e.g.,by the “provide audio file for prompts” 11320 module) the directionalityof the prompt and either creates an audio file or chooses an audio filefrom the audio files for prompt storage 11330 where the inherentproperties authored into the audio file create the panning from onestereo channel to the other. In these embodiments, the “provide audiofile for prompts” 11320 module does not send the prompt directionalityto the audio system and instead sends an audio file that provides thepanning effect when played by the audio system. These embodiments alsodo not utilize the “set pan level” 11350 module.

VII. Electronic System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium). When these instructions are executed by one or morecomputational or processing unit(s) (e.g., one or more processors, coresof processors, or other processing units), they cause the processingunit(s) to perform the actions indicated in the instructions. Examplesof computer readable media include, but are not limited to, CD-ROMs,flash drives, random access memory (RAM) chips, hard drives, erasableprogrammable read-only memories (EPROMs), electrically erasableprogrammable read-only memories (EEPROMs), etc. The computer readablemedia does not include carrier waves and electronic signals passingwirelessly or over wired connections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storagewhich can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs. Finally, any combination ofseparate programs that together implement a software invention describedhere is within the scope of the invention. In some embodiments, thesoftware programs, when installed to operate on one or more electronicsystems, define one or more specific machine implementations thatexecute and perform the operations of the software programs.

A. Mobile Device

The mapping and navigation applications of some embodiments operate onmobile devices, such as smart phones (e.g., iPhones®) and tablets (e.g.,iPads®). FIG. 114 is an example of an architecture 11400 of such amobile computing device. Examples of mobile computing devices includesmartphones, tablets, laptops, etc. As shown, the mobile computingdevice 11400 includes one or more processing units 11405, a memoryinterface 11410 and a peripherals interface 11415.

The peripherals interface 11415 is coupled to various sensors andsubsystems, including a camera subsystem 11420, a wireless communicationsubsystem(s) 11425, an audio subsystem 11430, an I/O subsystem 11435,etc. The peripherals interface 11415 enables communication between theprocessing units 11405 and various peripherals. For example, anorientation sensor 11445 (e.g., a gyroscope) and an acceleration sensor11450 (e.g., an accelerometer) is coupled to the peripherals interface11415 to facilitate orientation and acceleration functions.

The camera subsystem 11420 is coupled to one or more optical sensors11440 (e.g., a charged coupled device (CCD) optical sensor, acomplementary metal-oxide-semiconductor (CMOS) optical sensor, etc.).The camera subsystem 11420 coupled with the optical sensors 11440facilitates camera functions, such as image and/or video data capturing.The wireless communication subsystem 11425 serves to facilitatecommunication functions. In some embodiments, the wireless communicationsubsystem 11425 includes radio frequency receivers and transmitters, andoptical receivers and transmitters (not shown in FIG. 114). Thesereceivers and transmitters of some embodiments are implemented tooperate over one or more communication networks such as a GSM network, aWi-Fi network, a Bluetooth network, etc. The audio subsystem 11430 iscoupled to a speaker to output audio (e.g., to output voice navigationinstructions). Additionally, the audio subsystem 11430 is coupled to amicrophone to facilitate voice-enabled functions, such as voicerecognition (e.g., for searching), digital recording, etc.

The I/O subsystem 11435 involves the transfer between input/outputperipheral devices, such as a display, a touch screen, etc., and thedata bus of the processing units 11405 through the peripherals interface11415. The I/O subsystem 11435 includes a touch-screen controller 11455and other input controllers 11460 to facilitate the transfer betweeninput/output peripheral devices and the data bus of the processing units11405. As shown, the touch-screen controller 11455 is coupled to a touchscreen 11465. The touch-screen controller 11455 detects contact andmovement on the touch screen 11465 using any of multiple touchsensitivity technologies. The other input controllers 11460 are coupledto other input/control devices, such as one or more buttons. Someembodiments include a near-touch sensitive screen and a correspondingcontroller that can detect near-touch interactions instead of or inaddition to touch interactions.

The memory interface 11410 is coupled to memory 11470. In someembodiments, the memory 11470 includes volatile memory (e.g., high-speedrandom access memory), non-volatile memory (e.g., flash memory), acombination of volatile and non-volatile memory, and/or any other typeof memory. As illustrated in FIG. 114, the memory 11470 stores anoperating system (OS) 11472. The OS 11472 includes instructions forhandling basic system services and for performing hardware dependenttasks.

The memory 11470 also includes communication instructions 11474 tofacilitate communicating with one or more additional devices; graphicaluser interface instructions 11476 to facilitate graphic user interfaceprocessing; image processing instructions 11478 to facilitateimage-related processing and functions; input processing instructions11480 to facilitate input-related (e.g., touch input) processes andfunctions; audio processing instructions 11482 to facilitateaudio-related processes and functions; and camera instructions 11484 tofacilitate camera-related processes and functions. The instructionsdescribed above are merely exemplary and the memory 11470 includesadditional and/or other instructions in some embodiments. For instance,the memory for a smartphone may include phone instructions to facilitatephone-related processes and functions. Additionally, the memory mayinclude instructions for a mapping and navigation application as well asother applications. The above-identified instructions need not beimplemented as separate software programs or modules. Various functionsof the mobile computing device can be implemented in hardware and/or insoftware, including in one or more signal processing and/or applicationspecific integrated circuits.

While the components illustrated in FIG. 114 are shown as separatecomponents, one of ordinary skill in the art will recognize that two ormore components may be integrated into one or more integrated circuits.In addition, two or more components may be coupled together by one ormore communication buses or signal lines. Also, while many of thefunctions have been described as being performed by one component, oneof ordinary skill in the art will realize that the functions describedwith respect to FIG. 114 may be split into two or more integratedcircuits.

B. Computer System

FIG. 115 conceptually illustrates another example of an electronicsystem 11500 with which some embodiments of the invention areimplemented. The electronic system 11500 may be a computer (e.g., adesktop computer, personal computer, tablet computer, etc.), phone, PDA,or any other sort of electronic or computing device. Such an electronicsystem includes various types of computer readable media and interfacesfor various other types of computer readable media. Electronic system11500 includes a bus 11505, processing unit(s) 11510, a graphicsprocessing unit (GPU) 11515, a system memory 11520, a network 11525, aread-only memory 11530, a permanent storage device 11535, input devices11540, and output devices 11545.

The bus 11505 collectively represents all system, peripheral, andchipset buses that communicatively connect the numerous internal devicesof the electronic system 11500. For instance, the bus 11505communicatively connects the processing unit(s) 11510 with the read-onlymemory 11530, the GPU 11515, the system memory 11520, and the permanentstorage device 11535.

From these various memory units, the processing unit(s) 11510 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Someinstructions are passed to and executed by the GPU 11515. The GPU 11515can offload various computations or complement the image processingprovided by the processing unit(s) 11510. In some embodiments, suchfunctionality can be provided using CoreImage's kernel shading language.

The read-only-memory (ROM) 11530 stores static data and instructionsthat are needed by the processing unit(s) 11510 and other modules of theelectronic system. The permanent storage device 11535, on the otherhand, is a read-and-write memory device. This device is a non-volatilememory unit that stores instructions and data even when the electronicsystem 11500 is off. Some embodiments of the invention use amass-storage device (such as a magnetic or optical disk and itscorresponding disk drive, integrated flash memory) as the permanentstorage device 11535.

Other embodiments use a removable storage device (such as a floppy disk,flash memory device, etc., and its corresponding drive) as the permanentstorage device. Like the permanent storage device 11535, the systemmemory 11520 is a read-and-write memory device. However, unlike storagedevice 11535, the system memory 11520 is a volatile read-and-writememory, such a random access memory. The system memory 11520 stores someof the instructions and data that the processor needs at runtime. Insome embodiments, the invention's processes are stored in the systemmemory 11520, the permanent storage device 11535, and/or the read-onlymemory 11530. For example, the various memory units include instructionsfor processing multimedia clips in accordance with some embodiments.From these various memory units, the processing unit(s) 11510 retrievesinstructions to execute and data to process in order to execute theprocesses of some embodiments.

The bus 11505 also connects to the input and output devices 11540 and11545. The input devices 11540 enable the user to communicateinformation and select commands to the electronic system. The inputdevices 11540 include alphanumeric keyboards and pointing devices (alsocalled “cursor control devices”), cameras (e.g., webcams), microphonesor similar devices for receiving voice commands, etc. The output devices11545 display images generated by the electronic system or otherwiseoutput data. The output devices 11545 include printers and displaydevices, such as cathode ray tubes (CRT) or liquid crystal displays(LCD), as well as speakers or similar audio output devices. Someembodiments include devices such as a touchscreen that function as bothinput and output devices.

Finally, as shown in FIG. 115, bus 11505 also couples electronic system11500 to a network 11525 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or anIntranet), or a network of networks, such as the Internet. Any or allcomponents of electronic system 11500 may be used in conjunction withthe invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself. In addition, someembodiments execute software stored in programmable logic devices(PLDs), ROM, or RAM devices.

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium,” “computer readable media,” and “machinereadable medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by a computer. Theseterms exclude any wireless signals, wired download signals, and anyother ephemeral signals.

VIII. Map Service Environment

Various embodiments may operate within a map service operatingenvironment. FIG. 116 illustrates a map service operating environment,according to some embodiments. A map service 11630 (also referred to asmapping service) may provide map services for one or more client devices11602 a-11602 c in communication with the map service 11630 throughvarious communication methods and protocols. A map service 11630 in someembodiments provides map information and other map-related data, such astwo-dimensional map image data (e.g., aerial view of roads utilizingsatellite imagery), three-dimensional map image data (e.g., traversablemap with three-dimensional features, such as buildings), route anddirection calculations (e.g., ferry route calculations or directionsbetween two points for a pedestrian), real-time navigation data (e.g.,turn-by-turn visual navigation data in two or three dimensions),location data (e.g., where the client device is currently located), andother geographic data (e.g., wireless network coverage, weather, trafficinformation, or nearby points-of-interest). In various embodiments, themap service data may include localized labels for different countries orregions. Localized labels may be utilized to present map labels (e.g.,street names, city names, points of interest) in different languages onclient devices. Client devices 11602 a-11602 c may utilize these mapservices by obtaining map service data. Client devices 11602 a-11602 cmay implement various techniques to process map service data. Clientdevices 11602 a-11602 c may then provide map services to variousentities, including, but not limited to, users, internal software orhardware modules, and/or other systems or devices external to the clientdevices 11602 a-11602 c.

In some embodiments, a map service is implemented by one or more nodesin a distributed computing system. Each node may be assigned one or moreservices or components of a map service. Some nodes may be assigned thesame map service or component of a map service. A load balancing node insome embodiments distributes access or requests to other nodes within amap service. In some embodiments a map service is implemented as asingle system, such as a single server. Different modules or hardwaredevices within a server may implement one or more of the variousservices provided by a map service.

A map service in some embodiments provides map services by generatingmap service data in various formats. In some embodiments, one format ofmap service data is map image data. Map image data provides image datato a client device so that the client device may process the image data(e.g., rendering and/or displaying the image data as a two-dimensionalor three-dimensional map). Map image data, whether in two or threedimensions, may specify one or more map tiles. A map tile may be aportion of a larger map image. Assembling together the map tiles of amap produces the original map. Tiles may be generated from map imagedata, routing or navigation data, or any other map service data. In someembodiments map tiles are raster-based map tiles, with tile sizesranging from any size both larger and smaller than a commonly-used 256pixel by 256 pixel tile. Raster-based map tiles may be encoded in anynumber of standard digital image representations including, but notlimited to, Bitmap (.bmp), Graphics Interchange Format (.gif), JointPhotographic Experts Group (.jpg, .jpeg, etc.), Portable NetworksGraphic (.png), or Tagged Image File Format (.tiff). In someembodiments, map tiles are vector-based map tiles, encoded using vectorgraphics, including, but not limited to, Scalable Vector Graphics (.svg)or a Drawing File (.drw). Some embodiments also include tiles with acombination of vector and raster data. Metadata or other informationpertaining to the map tile may also be included within or along with amap tile, providing further map service data to a client device. Invarious embodiments, a map tile is encoded for transport utilizingvarious standards and/or protocols, some of which are described inexamples below.

In various embodiments, map tiles may be constructed from image data ofdifferent resolutions depending on zoom level. For instance, for lowzoom level (e.g., world or globe view), the resolution of map or imagedata need not be as high relative to the resolution at a high zoom level(e.g., city or street level). For example, when in a globe view, theremay be no need to render street level artifacts as such objects would beso small as to be negligible in many cases.

A map service in some embodiments performs various techniques to analyzea map tile before encoding the tile for transport. This analysis mayoptimize map service performance for both client devices and a mapservice. In some embodiments map tiles are analyzed for complexity,according to vector-based graphic techniques, and constructed utilizingcomplex and non-complex layers. Map tiles may also be analyzed forcommon image data or patterns that may be rendered as image textures andconstructed by relying on image masks. In some embodiments, raster-basedimage data in a map tile contains certain mask values, which areassociated with one or more textures. Some embodiments also analyze maptiles for specified features that may be associated with certain mapstyles that contain style identifiers.

Other map services generate map service data relying upon various dataformats separate from a map tile in some embodiments. For instance, mapservices that provide location data may utilize data formats conformingto location service protocols, such as, but not limited to, RadioResource Location services Protocol (RRLP), TIA 801 for Code DivisionMultiple Access (CDMA), Radio Resource Control (RRC) position protocol,or LTE Positioning Protocol (LPP). Embodiments may also receive orrequest data from client devices identifying device capabilities orattributes (e.g., hardware specifications or operating system version)or communication capabilities (e.g., device communication bandwidth asdetermined by wireless signal strength or wire or wireless networktype).

A map service may obtain map service data from internal or externalsources. For example, satellite imagery used in map image data may beobtained from external services, or internal systems, storage devices,or nodes. Other examples may include, but are not limited to, GPSassistance servers, wireless network coverage databases, business orpersonal directories, weather data, government information (e.g.,construction updates or road name changes), or traffic reports. Someembodiments of a map service may update map service data (e.g., wirelessnetwork coverage) for analyzing future requests from client devices.

Various embodiments of a map service may respond to client devicerequests for map services. These requests may be for a specific maps orportions of a map. Some embodiments format requests for a map asrequests for certain map tiles. In some embodiments, requests alsosupply the map service with starting locations (or current locations)and destination locations for a route calculation. A client device mayalso request map service rendering information, such as map textures orstyle sheets. In at least some embodiments, requests are also one of aseries of requests implementing turn-by-turn navigation. Requests forother geographic data may include, but are not limited to, requests forcurrent location, wireless network coverage, weather, trafficinformation, or nearby points-of-interest.

A map service, in some embodiments, analyzes client device requests tooptimize a device or map service operation. For instance, a map servicemay recognize that the location of a client device is in an area of poorcommunications (e.g., weak wireless signal) and send more map servicedata to supply a client device in the event of loss in communication orsend instructions to utilize different client hardware (e.g.,orientation sensors) or software (e.g., utilize wireless locationservices or Wi-Fi positioning instead of GPS-based services). In anotherexample, a map service may analyze a client device request forvector-based map image data and determine that raster-based map databetter optimizes the map image data according to the image's complexity.Embodiments of other map services may perform similar analysis on clientdevice requests and, as such, the above examples are not intended to belimiting.

Various embodiments of client devices (e.g., client devices 11602a-11602 c) are implemented on different portable-multifunction devicetypes. Client devices 11602 a-11602 c utilize map service 11630 throughvarious communication methods and protocols. In some embodiments, clientdevices 11602 a-11602 c obtain map service data from map service 11630.Client devices 11602 a-11602 c request or receive map service data.Client devices 11602 a-11602 c then process map service data (e.g.,render and/or display the data) and may send the data to anothersoftware or hardware module on the device or to an external device orsystem.

A client device, according to some embodiments, implements techniques torender and/or display maps. These maps may be requested or received invarious formats, such as map tiles described above. A client device mayrender a map in two-dimensional or three-dimensional views. Someembodiments of a client device display a rendered map and allow a user,system, or device providing input to manipulate a virtual camera in themap, changing the map display according to the virtual camera'sposition, orientation, and field-of-view. Various forms and inputdevices are implemented to manipulate a virtual camera. In someembodiments, touch input, through certain single or combination gestures(e.g., touch-and-hold or a swipe) manipulate the virtual camera. Otherembodiments allow manipulation of the device's physical location tomanipulate a virtual camera. For instance, a client device may be tiltedup from its current position to manipulate the virtual camera to rotateup. In another example, a client device may be tilted forward from itscurrent position to move the virtual camera forward. Other input devicesto the client device may be implemented including, but not limited to,auditory input (e.g., spoken words), a physical keyboard, mouse, and/ora joystick.

Some embodiments provide various visual feedback to virtual cameramanipulations, such as displaying an animation of possible virtualcamera manipulations when transitioning from two-dimensional map viewsto three-dimensional map views. Some embodiments also allow input toselect a map feature or object (e.g., a building) and highlight theobject, producing a blur effect that maintains the virtual camera'sperception of three-dimensional space.

In some embodiments, a client device implements a navigation system(e.g., turn-by-turn navigation). A navigation system provides directionsor route information, which may be displayed to a user. Some embodimentsof a client device request directions or a route calculation from a mapservice. A client device may receive map image data and route data froma map service. In some embodiments, a client device implements aturn-by-turn navigation system, which provides real-time route anddirection information based upon location information and routeinformation received from a map service and/or other location system,such as a Global Positioning Satellite (GPS). A client device maydisplay map image data that reflects the current location of the clientdevice and update the map image data in real-time. A navigation systemmay provide auditory or visual directions to follow a certain route.

A virtual camera is implemented to manipulate navigation map dataaccording to some embodiments. In some embodiments, the client devicesallow the device to adjust the virtual camera display orientation tobias toward the route destination. Some embodiments also allow thevirtual camera to navigate turns by simulating the inertial motion ofthe virtual camera.

Client devices implement various techniques to utilize map service datafrom map service. Some embodiments implement some techniques to optimizerendering of two-dimensional and three-dimensional map image data. Insome embodiments, a client device locally stores rendering information.For instance, a client stores a style sheet, which provides renderingdirections for image data containing style identifiers. In anotherexample, common image textures may be stored to decrease the amount ofmap image data transferred from a map service. Client devices indifferent embodiments implement various modeling techniques to rendertwo-dimensional and three-dimensional map image data, examples of whichinclude, but are not limited to: generating three-dimensional buildingsout of two-dimensional building footprint data; modeling two-dimensionaland three-dimensional map objects to determine the client devicecommunication environment; generating models to determine whether maplabels are seen from a certain virtual camera position; and generatingmodels to smooth transitions between map image data. In someembodiments, the client devices also order or prioritize map servicedata in certain techniques. For instance, a client device detects themotion or velocity of a virtual camera, which if exceeding certainthreshold values, lower-detail image data is loaded and rendered forcertain areas. Other examples include: rendering vector-based curves asa series of points, preloading map image data for areas of poorcommunication with a map service, adapting textures based on displayzoom level, or rendering map image data according to complexity.

In some embodiments, client devices communicate utilizing various dataformats separate from a map tile. For instance, some client devicesimplement Assisted Global Positioning Satellites (A-GPS) and communicatewith location services that utilize data formats conforming to locationservice protocols, such as, but not limited to, Radio Resource Locationservices Protocol (RRLP), TIA 801 for Code Division Multiple Access(CDMA), Radio Resource Control (RRC) position protocol, or LTEPositioning Protocol (LPP). Client devices may also receive GPS signalsdirectly. Embodiments may also send data, with or without solicitationfrom a map service, identifying the client device's capabilities orattributes (e.g., hardware specifications or operating system version)or communication capabilities (e.g., device communication bandwidth asdetermined by wireless signal strength or wire or wireless networktype).

FIG. 116 illustrates one possible embodiment of an operating environment11600 for a map service 11630 and client devices 11602 a-11602 c. Insome embodiments, devices 11602 a, 11602 b, and 11602 c communicate overone or more wire or wireless networks 11610. For example, wirelessnetwork 11610, such as a cellular network, can communicate with a widearea network (WAN) 11620, such as the Internet, by use of gateway 11614.A gateway 11614 in some embodiments provides a packet oriented mobiledata service, such as General Packet Radio Service (GPRS), or othermobile data service allowing wireless networks to transmit data to othernetworks, such as wide area network 11620. Likewise, access device 11612(e.g., IEEE 802.11g wireless access device) provides communicationaccess to WAN 11620. Devices 11602 a and 11602 b can be any portableelectronic or computing device capable of communicating with a mapservice. Device 11602 c can be any non-portable electronic or computingdevice capable of communicating with a map service.

In some embodiments, both voice and data communications are establishedover wireless network 11610 and access device 11612. For instance,device 11602 a can place and receive phone calls (e.g., using voice overInternet Protocol (VoIP) protocols), send and receive e-mail messages(e.g., using Simple Mail Transfer Protocol (SMTP) or Post OfficeProtocol 3 (POP3)), and retrieve electronic documents and/or streams,such as web pages, photographs, and videos, over wireless network 11610,gateway 11614, and WAN 11620 (e.g., using Transmission ControlProtocol/Internet Protocol (TCP/IP) or User Datagram Protocol (UDP)).Likewise, in some implementations, devices 11602 b and 11602 c can placeand receive phone calls, send and receive e-mail messages, and retrieveelectronic documents over access device 11612 and WAN 11620. In variousembodiments, any of the illustrated client devices may communicate withmap service 11630 and/or other service(s) 11650 using a persistentconnection established in accordance with one or more securityprotocols, such as the Secure Sockets Layer (SSL) protocol or theTransport Layer Security (TLS) protocol.

Devices 11602 a and 11602 b can also establish communications by othermeans. For example, wireless device 11602 a can communicate with otherwireless devices (e.g., other devices 11602 b, cell phones, etc.) overthe wireless network 11610. Likewise devices 11602 a and 11602 b canestablish peer-to-peer communications 11640 (e.g., a personal areanetwork) by use of one or more communication subsystems, such asBluetooth® communication from Bluetooth Special Interest Group, Inc. ofKirkland, Wash. Device 11602 c can also establish peer to peercommunications with devices 11602 a or 11602 b (not shown). Othercommunication protocols and topologies can also be implemented. Devices11602 a and 11602 b may also receive Global Positioning Satellite (GPS)signals from GPS satellites 11660.

Devices 11602 a, 11602 b, and 11602 c can communicate with map service11630 over one or more wired and/or wireless networks, 11612 or 11610.For instance, map service 11630 can provide map service data torendering devices 11602 a, 11602 b, and 11602 c. Map service 11630 mayalso communicate with other services 11650 to obtain data to implementmap services. Map service 11630 and other services 11650 may alsoreceive GPS signals from GPS satellites 11660.

In various embodiments, map service 11630 and/or other service(s) 11650are configured to process search requests from any of the clientdevices. Search requests may include but are not limited to queries forbusinesses, addresses, residential locations, points of interest, orsome combination thereof. Map service 11630 and/or other service(s)11650 may be configured to return results related to a variety ofparameters including but not limited to a location entered into anaddress bar or other text entry field (including abbreviations and/orother shorthand notation), a current map view (e.g., user may be viewingone location on the multifunction device while residing in anotherlocation), current location of the user (e.g., in cases where thecurrent map view did not include search results), and the current route(if any). In various embodiments, these parameters may affect thecomposition of the search results (and/or the ordering of the searchresults) based on different priority weightings. In various embodiments,the search results that are returned may be a subset of results selectedbased on specific criteria including but not limited to a quantity oftimes the search result (e.g., a particular point of interest) has beenrequested, a measure of quality associated with the search result (e.g.,highest user or editorial review rating), and/or the volume of reviewsfor the search results (e.g., the number of times the search result hasbeen review or rated).

In various embodiments, map service 11630 and/or other service(s) 11650are configured to provide auto-complete search results that aredisplayed on the client device, such as within the mapping application.For instance, auto-complete search results may populate a portion of thescreen as the user enters one or more search keywords on themultifunction device. In some cases, this feature may save the user timeas the desired search result may be displayed before the user enters thefull search query. In various embodiments, the auto complete searchresults may be search results found by the client on the client device(e.g., bookmarks or contacts), search results found elsewhere (e.g.,from the Internet) by map service 11630 and/or other service(s) 11650,and/or some combination thereof. As is the case with commands, any ofthe search queries may be entered by the user via voice or throughtyping. The multifunction device may be configured to display searchresults graphically within any of the map display described herein. Forinstance, a pin or other graphical indicator may specify locations ofsearch results as points of interest. In various embodiments, responsiveto a user selection of one of these points of interest (e.g., a touchselection, such as a tap), the multifunction device is configured todisplay additional information about the selected point of interestincluding but not limited to ratings, reviews or review snippets, hoursof operation, store status (e.g., open for business, permanently closed,etc.), and/or images of a storefront for the point of interest. Invarious embodiments, any of this information may be displayed on agraphical information card that is displayed in response to the user'sselection of the point of interest.

In various embodiments, map service 11630 and/or other service(s) 11650provide one or more feedback mechanisms to receive feedback from clientdevices 11602 a-11602 c. For instance, client devices may providefeedback on search results to map service 11630 and/or other service(s)11650 (e.g., feedback specifying ratings, reviews, temporary orpermanent business closures, errors etc.); this feedback may be used toupdate information about points of interest in order to provide moreaccurate or more up-to-date search results in the future. In someembodiments, map service 11630 and/or other service(s) 11650 may providetesting information to the client device (e.g., an A/B test) todetermine which search results are best. For instance, at randomintervals, the client device may receive and present two search resultsto a user and allow the user to indicate the best result. The clientdevice may report the test results to map service 11630 and/or otherservice(s) 11650 to improve future search results based on the chosentesting technique, such as an A/B test technique in which a baselinecontrol sample is compared to a variety of single-variable test samplesin order to improve results.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. For instance, many of the figuresillustrate various touch gestures (e.g., taps, double taps, swipegestures, press and hold gestures, etc.). However, many of theillustrated operations could be performed via different touch gestures(e.g., a swipe instead of a tap, etc.) or by non-touch input (e.g.,using a cursor controller, a keyboard, a touchpad/trackpad, a near-touchsensitive screen, etc.). In addition, a number of the figuresconceptually illustrate processes. The specific operations of theseprocesses may not be performed in the exact order shown and described.The specific operations may not be performed in one continuous series ofoperations, and different specific operations may be performed indifferent embodiments. Furthermore, the process could be implementedusing several sub-processes, or as part of a larger macro process.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. In addition, a number of the figuresconceptually illustrate processes. The specific operations of theseprocesses may not be performed in the exact order shown and described.The specific operations may not be performed in one continuous series ofoperations, and different specific operations may be performed indifferent embodiments. Furthermore, the process could be implementedusing several sub-processes, or as part of a larger macro process. Thus,one of ordinary skill in the art would understand that the invention isnot to be limited by the foregoing illustrative details, but rather isto be defined by the appended claims.

1. (canceled)
 2. A method performed by a mobile device, the methodcomprising: determining by a navigation application that an audio promptshould be presented by the mobile device; receiving by the mobile devicethe audio prompt from the navigation application, the audio prompt to bepresented by the mobile device; determining that a current audio outputis currently being presented by the mobile device; determining an audiooutput type corresponding to the current audio output; when the audiooutput type corresponds to a first type, suspending the current audiooutput while presenting the audio prompt; and when the audio output typecorresponds to a second type, ducking the current audio output whilepresenting the audio prompt.
 3. The method of claim 2, wherein the firsttype comprises spoken words.
 4. The method of claim 2, wherein thesecond type comprises music.
 5. The method of claim 2, wherein duckingthe current audio output while presenting the audio prompt includesreducing a first volume of the current audio output to a second volumewithout muting the current audio output while presenting the audioprompt at a third volume that is louder than the second volume.
 6. Themethod of claim 5, further comprising: automatically increasing thesecond volume of the current audio output to the first volume uponcompleting the presentation of the audio prompt.
 7. The method of claim2, further comprising: automatically resuming the current audio outputupon completing the presentation of the audio prompt.
 8. The method ofclaim 2, wherein the audio prompt comprises a verbal navigationalinstruction.
 9. A non-transitory computer readable medium including oneor more sequences of instructions that, when executed by one or moreprocessors, cause the processors to perform operations comprising:determining by a navigation application that an audio prompt should bepresented by a mobile device; receiving by the mobile device the audioprompt from the navigation application, the audio prompt to be presentedby the mobile device; determining that a current audio output iscurrently being presented by the mobile device; determining an audiooutput type corresponding to the current audio output; when the audiooutput type corresponds to a first type, suspending the current audiooutput while presenting the audio prompt; and when the audio output typecorresponds to a second type, ducking the current audio output whilepresenting the audio prompt.
 10. The non-transitory computer readablemedium of claim 9, wherein the first type comprises spoken words. 11.The non-transitory computer readable medium of claim 9, wherein thesecond type comprises music.
 12. The non-transitory computer readablemedium of claim 9, wherein ducking the current audio output whilepresenting the audio prompt includes reducing a first volume of thecurrent audio output to a second volume without muting the current audiooutput while presenting the audio prompt at a third volume that islouder than the second volume.
 13. The non-transitory computer readablemedium of claim 12, wherein the instructions cause: automaticallyincreasing the second volume of the current audio output to the firstvolume upon completing the presentation of the audio prompt.
 14. Thenon-transitory computer readable medium of claim 9, wherein theinstructions cause: automatically resuming the current audio output uponcompleting the presentation of the audio prompt.
 15. The non-transitorycomputer readable medium of claim 9, wherein the audio prompt comprisesa verbal navigational instruction.
 16. A system comprising: one or moreprocessors; and a non-transitory computer readable medium including oneor more sequences of instructions that, when executed by the one or moreprocessors, cause the processors to perform operations comprising:determining that an audio prompt should be presented by a mobile device;determining that a current audio output is currently being presented bythe mobile device; determining an audio output type corresponding to thecurrent audio output; when the audio output type corresponds to a firsttype, suspending the current audio output while presenting the audioprompt; and when the audio output type corresponds to a second type,ducking the current audio output while presenting the audio prompt. 17.The system of claim 16, wherein the first type comprises spoken words.18. The system of claim 16, wherein the second type comprises music. 19.The system of claim 16, wherein ducking the current audio output whilepresenting the audio prompt includes reducing a first volume of thecurrent audio output to a second volume without muting the current audiooutput while presenting the audio prompt at a third volume that islouder than the second volume.
 20. The system of claim 19, wherein theinstructions cause: automatically increasing the second volume of thecurrent audio output to the first volume upon completing thepresentation of the audio prompt.
 21. The system of claim 16, whereinthe instructions cause: automatically resuming the current audio outputupon completing the presentation of the audio prompt.
 22. The system ofclaim 16, wherein the audio prompt comprises a verbal navigationalinstruction.